`basilisp.core.async` namespace

[chrisrink10]

[chrisrink10]

The functionality of core.async depends primarily on the channel primitive. The package aiochan implements the channel primitive using Python's builtin async/await functionality. This would be a super rad enhancement.

[FalseProtagonist]

Hi,

maybe this will be helpful:
I've done ~as you suggested, using aiochan + async/await to have something /like/ core.async in my own code

a significant consideration/design element is:

• in python, running an event loop blocks a thread, so only one can be "live" per thread
• in clojure, async go blocks use a fixed thread-pool and so operate independently, "go" is shipping stuff to another thread
• aiochan is event-loop async/await oriented not thread-oriented
• if we use pure event-loop implementation, it looks more like cljs where everything is sharing one thread
  • but in order to run an event loop we need to block a thread
  • threads can't (simply) share event loops
• what I'm currently doing is defining an event-loop/thread dedicated on app startup, putting it in a top-level var and passing it around, then using run_coroutine_threadsafe (inside my go macro) to push async things into my dedicated event-loop-thread

This was fairly simple to implement but isn't allowing parallelism, unlike clojure go-blocks do by default, and somehow seems unclojuric, maybe better to run a thread per go block a-la clojure, or maybe given GIL a process. THen would need to work out how to create a thread/process-safe channel implementation though, or hack aiochan.

[amano-kenji]

I propose the following synchronous API for dealing with python async constructs. I want core.async to synchronize async API. It may also have asyncio-compatible API which wouldn't need the following extra functions.

• asyncio.run_coroutine_threadsafe(c, l).add_callback_fn((fn [p] (go (>! ch (.result p)))))
  • basilisp.core.async.interop/coroutine turns a coroutine into an unbuffered channel.
  • basilisp.core.async.interop/async-iter turns an async iterator into an unbuffered channel.

Notes

• We can have a separate thread that runs an asyncio event loop. The loop should be available internally inside basilisp.core.async.interop.
• The callback passed to add_callback_fn is called in the event loop thread. You don't want to block the event loop, so you create a go block in the callback.
• With <!!, you can extract a value from a coroutine. With <!! and lazy-seq, you can turn an async iterator into a lazy sequence.
• If python ever gains something like JVM virtual threads, we can consider promesa's core.async implementation that allows a go block to cross function boundaries. If we have JVM virtual threads on python VM, we can even ditch asyncio.
• CPU-bound operations will engage GIL. Python 3.13 has experimental support for free threading which disables GIL. I don't know when free threading will become the default. However, I just want to synchronize IO operations and GPU operations trapped in async context for now.
• go blocks should share one dedicated thread for now.
• https://pypi.org/project/cs-queues implements unbuffered channel. You can use it as a reference when you implement channels.

It took a while to come up with this design. Because my design proposals seem quite comprehensive, I think I can take a break for now.

[amano-kenji]

In https://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/, a man says the correct abstraction for dealing with asynchronism is (lightweight) threads.

Go blocks are lightweight threads analogous to goroutines, JVM virtual threads, lua coroutines, ruby fibers, and janet fibers.

The blog author didn't talk about channels or queues, but communication among (lightweight) threads with queues or channels has been the best concurrency programming experience. It's said that Go language has the best concurrency programming model because threads and goroutines just communicate with each other through channels. This makes code look synchronous. Lightweight threads + channels = the best concurrency programming model.

My j3blocks library utilizes janet fibers and janet channels extensively. Janet fibers can behave like core.async go blocks or lazy sequences. Janet channels are conceptually identical to core.async channels.