Support workflows for creating library components inside a Spin application using them

[tschneidereit]

In #2970, I'm adding a template for a new library component, to be consumed by other components in the same application. @itowlson rightfully raised the question of how that fits into the scope of a Spin application; I'm opening this issue to discuss that question separately.

For context, Spin's current support for dependencies is focused on using existing library components, commonly retrieved from a registry. That is very similar to using a published library from crates.io, npmjs.com, etc.

What we don't have established workflows for is another common use case in other ecosystems: developing a larger application (or library) in a workspace which consists of multiple "modules" (i.e., crates for cargo, packages for npm, etc.)

I want to propose explicitly considering this use case as in scope for Spin as well, and to identify changes needed to support it better.

The key motivation is that it's often useful to componentize an application without necessarily treating individual components as self-contained projects, ready for publishing to a registry on their own. In the context of the component model, typical motivations for this include wanting to use different languages for implementing different parts of an application, or reducing accessible capabilities for security-sensitive code.

I'd like to gather input on the overall goal here as a first step. Afterwards, we can start identifying changes derived from considering this goal as in scope for Spin, if that's the conclusion we arrive at.

[lann]

Ref (dupe?) #1759

[seungjin]

After being pretty much satisfied with this issue, my next challenge in making Spin my go-to platform for microservices is the following:

My main spin.toml file is getting bigger and harder to maintain – here’s the file.
Also, I need to maintain at least two spin.toml files for each microservice: one for the higher-level project that orchestrates all services, and another for each individual microservice (for dev and test purpose).

It seems like it's time for me to stop managing spin.toml files manually and start doing it programmatically. I'm thinking about building a kind of poor man's TOML merge system, but it would be custom and probably never standardized.

If SpinFramework provides a built-in solution that the community can share and use consistently, it would be greatly appreciated.

Do we have any update or status (like priority) on this issue?

[itowlson]

@seungjin Just to clarify, this issue is specifically about components used in composition rather that directly associated with a trigger - the large manifest problem you mention is maybe more addressed in #3073 - am I misunderstanding your comment?

[seungjin]

@seungjin Just to clarify, this issue is specifically about components used in composition rather that directly associated with a trigger - the large manifest problem you mention is maybe more addressed in #3073 - am I misunderstanding your comment?

Oh. You are right. I got a wrong address!