Skip to content

Errors, missing code/links in Workers section #179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Neon22 opened this issue Apr 5, 2025 · 5 comments
Open

Errors, missing code/links in Workers section #179

Neon22 opened this issue Apr 5, 2025 · 5 comments

Comments

@Neon22
Copy link
Contributor

Neon22 commented Apr 5, 2025

In one place where workers are mentioned:

There are several pieces of code with code elements missing.
Also the two ways workers can be started are confusing because of this missing code.
Also the link to a working example is 404

The only problem with this is that people who are NOT Javascript experts may not be able to work out all the usecases that are legitimate from the code available. (I know I can't.)

  1. Missing link:
  • In this section of the page is a link:
The values passed between the main thread and the worker must be serializable. Try the example given above via [this project on PyScript.com](https://pyscript.com/@ntoll/tiny-silence/latest).
  • this link is 404
  1. In the Section Common use case the code refers to greeting which is defined in the previous section Worker interactions but not in this code. Its not clear which code should be carried forward from example to example and which is standalone. Ideally all the code for main and worker should (IMHO) be standalone and complete - or point to an example which works.

IMHO - It would be nice if the sections showed:

  • starting a worker from index.html
    • that does one job and has one function (from pyscript import workers)?
    • same but can be called multiple times with multiple functions (__export__ = ["fname1", "fname2"])?
    • some of which update the dom directly and some of which pass data back to main.
  • starting from python using Pyworker (from pyscript import PyWorker)
    • defining multiple functions that can be called,
    • setting a config so a package can be loaded,
    • some of which update the dom directly and some of which pass data back to main.
  • whatever is required for the donkey to:
    • send new code to the donkey for it to operate with,
    • see results like donkey's main use case - running code and seeing the results in a textarea,
    • see results where it can do the jobs defined for the previous two cases (sending data back).

IMHO this would make it easier for people to work out how to use workers in pyscript.
Any examples that could be generated for the docs would also be great and it would be nice if they followed the categories indicated above.
(I'd happily contribute a demo if I could work out how to make it ALL work.)

@Neon22
Copy link
Contributor Author

Neon22 commented Apr 6, 2025

So using the example on the page above - and trying to create a worker from within main.py:

from pyscript import PyWorker
# The type MUST be given and can be either `micropython` or `pyodide`
my_worker = PyWorker("my-worker-code.py", type="micropython")

Creating worker from main

I change to pyodide and add config (which was hard to work out) to get:
But can I even add a config ? I guess I can because its pyodide not mpy right? but is the intepreter defined like this or redundant because I specified the type?

from pyscript import PyWorker
from pyscript.ffi import to_js
worker_config = {
       "interpreter": "pyodide/pyodide.js",
       "packages": [ ],
       "files": {"./satin_worker/worker.py" : "",
                   "./satin_worker/get_satins.py" : "",
                   "./satin_worker/weave_pattern.py" : ""}
        }
worker = PyWorker("worker.py", type="pyodide", config=ltk.to_js(worker_config))
print("before ready")
# Await until Pyodide has completed its bootstrap, and is ready.
await self.worker.ready
print("after ready")

With an empty file "./worker.py" this code runs and reports the two "ready states" on the pyscript terminal.
Even if "./satin_worker/worker.py" does not exist but ./worker.py does. No error reported.

If I change the code to:

worker = PyWorker("./satin_worker/worker.py", type="pyodide", config=to_js(worker_config))

I also get the two responses even if there is no "./satin_worker/worker.py"
The existence of the worker.py file seems ignored.

I suppose the environment starts but the worker has not been called yet - so it is not checked for existence.
but it is called because if I create the file and add two lines to it:

from pyscript import sync
print("Foo (onlyappears in JS console)")
sync.worker_msg("Bar")  # in terminalfile

Then "Foo (only appears in JS console)" will be printed into the JS console but not the pyscript terminal.
So now we know the system does not report the worker as missing in terminal or JS console.

and if main.py has one line added to it:
worker.sync.worker_msg = lambda var: print("Pyodide bootstrapped",var)
so now it looks like:

from pyscript import PyWorker
from pyscript.ffi import to_js
worker_config = {
       "interpreter": "pyodide/pyodide.js",
       "packages": [ ],
       "files": {"./satin_worker/worker.py" : "",
                   "./satin_worker/get_satins.py" : "",
                   "./satin_worker/weave_pattern.py" : ""}
        }
worker = PyWorker("worker.py", type="pyodide", config=to_js(worker_config))
worker.sync.worker_msg = lambda var: print("Pyodide bootstrapped",var)
print("before ready")
# Await until Pyodide has completed its bootstrap, and is ready.
await worker.ready
print("after ready")

Then I see in the pyscript terminal ""Pyodide bootstrapped Bar"
So now I know the worker is being started and called.

Doing something in the worker on request

So now to try to get the worker doing something after initialisation

If I add this function to main.py:

def worker_start_task(value):
    print("Initialising task", value)
    result = await worker.sync.long_task(value)
    print("Result:",result)

and in worker I add:

def long_task(value):
    return [1,2,value]

# Define what to export to the main thread.
__export__ = ["long_task"]

I cannot get worker_start_task("Baz") to print or do anything.

The docs seem to confuse starting a worker in index.html with starting a worker in main.py
These seem different enough to need their own sections.
The absence of any error messages in the JS console or terminal makes it hard to work out what is not happening.

@WebReflection
Copy link
Contributor

the sync is something you need to import and orchestrate, it's not based on each module exports because that would easily cause collisions between modules exported name ... you are better off with Polyscript documentation if you want to use internals or less documented parts, we have named workers to do what you are after, here it looks like you want to use internal bricks but, most importantly, there is a reason PyWorker is poorly documented, we actually believed we should remove it as it causes all the issues you are after because people use it to do things they might not fully understand so, at that point, use named workers or just workers as scripts on the page. The dance/orchestration is not trivial and at the end of the day those utilities are limited in features compared to the donkey, exported by the JS module, or the XWorker itself exported by Polyscript.

@Neon22
Copy link
Contributor Author

Neon22 commented Apr 8, 2025

I agree completely but I am using pyscript docs page.
So this page needs (IMHO) to be updated to be how we should use it.
I can't find docs on how to use it properly

@Neon22
Copy link
Contributor Author

Neon22 commented Apr 8, 2025

BTW the reason why I picked PyWorker from the docs is I want to use a worker in my single file components - so that I can make larger UI's out of files, which do the UI for one aspect of what is on the page.
If I can encapsulate the work that needs to be done into one file, and not have it require globals, then I can have isolation between each component and they won't affect each other.

If that's not a goal for pyscript - that's ok, Just means I won't be able to do that. I will then find another way to do it.
My work on this is documented in my PR for pyscript examples: pyscript/examples#26

@Neon22
Copy link
Contributor Author

Neon22 commented Apr 8, 2025

I have just found a section in the docs on named workers. Thanks for your hint.

I will see if this helps me do what I want. Cheers

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants