Library Guide: Add SQL level user guide:

[alamb]

Is your feature request related to a problem or challenge?

Part of #7014

If we want to have DataFusion used as the core of many new systems, we need it to be as easy as possible for someone to get their idea working on top of DataFusion.

Thanks to @tshauck we now have a basic Library Users Guide ❤️ and this ticket describes expanding it out

Describe the solution you'd like

Fill in the content of https://arrow.apache.org/datafusion/library-user-guide/using-the-sql-api.html

Example outline

1. Generating LogicalPlans from sql
2. Example / guide of how to extend SQL (both links to user defined functions and custom nodes, but also maybe some links to the datafusion sql variant parser
3. Example of how to run SQL read only and how to run it with catalog modifications (Misleading/invalid documentation on SessionContext::sql and SessionConfig::create_logical_plan #7272)

The examples directory holds a bunch of examples: https://github.com/apache/arrow-datafusion/tree/main/datafusion-examples

Describe alternatives you've considered

No response

Additional context

No response

[qstommyshu]

Hi, I can try to add some documentation for SQL level user guide as I'm exploring the project.

[alamb]

Thaks @qstommyshu

Something that that would be super helpful would be to move some of the basic examples from the examples directory into the documentation. Previously the code in examples was not tested but it is now.

For example, moving this example might be a great place to start

• https://github.com/apache/datafusion/blob/main/datafusion-examples/examples/sql_query.rs

And remove that example function and add it as examples in https://datafusion.apache.org/library-user-guide/using-the-sql-api.html

[qstommyshu]

Sounds good, I'll start with the example then!

[qstommyshu]

Hi @alamb

I've read through some examples in main/datafusion-examples/examples/, they are really helpful for understanding how to use datafusion.

And I've checked and tested examples in library user guide, the examples are correct and runnable (I copied the examples from doc and pasted it to sql_query.rs, everything runs flawlessly).

I made some code refactoring to the examples in /datafusion-examples/examples/sql_query.rs on local just to make it easier for a beginner to learn about datafusion. Maybe I can create a PR from here?

That being said, I think we probably don't need to remove example functions in the library user guide.

Fill in the content of arrow.apache.org/datafusion/library-user-guide/using-the-sql-api.html

I'll try to fill out the documentation content as I'm exploring the project.