Import subcollection data/ auto create subtable & array-subtable schemas

[shamsmosowi]

Currently, it's only possible to import collection data and auto generate schema for primary tables

Rowy should be able to detect and suggest the schema for both subtables and array subtables

The experience should be the same as the importing data/schema for primary collections

[rishipurwar1]

@shamsmosowi working on this issue!

[rishipurwar1]

Rowy should be able to detect and suggest the schema for both subtables and array subtables

Hello @shamsmosowi I did some research on how we're fetching the data from firestore for each collection:

1. In TableSourceFirestore.js, we call the useFirestoreCollectionAtom hook to fetch the docs from firestore and save them to tableRowsDbAtom.
2. Then, we access the tableRowsDbAtom in the table.ts file, which contains all the documents for a particular collection.
3. Then we merge the two arrays after removing the duplicates and update the value of tableRowsAtom.
4. Finally, we access the tableRowsAtom in the EmptyTable component to show the user that you already have some data in the firestore.

Am I right up to this point?

Now, to retrieve data for subcollections, we can use the same approach as we do for collections since fetching data for collections (collection(firebaseDb, pathToCollection)) and subcollections (collection(firebaseDb, pathToSubCollection)) works in the same way. However, we need to provide a way for the user to input the names of the subcollections, as each document can have multiple subcollections.

Approaches:

1. Provide a new field to the user at the time of filling out the "Create table" form just below the "Collection name" field.
2. Add one more tab in the "import" modal component to enter the subcollection names.

Another consideration is how we should present the subcollection data. For example, should we generate separate table pages for each subcollection, and how will users access these specific subcollection tables?

And Could you please tell me what you mean by "subtables" and "array subtables"?

Please let me know if you have any further thoughts or if there are any other aspects we should consider regarding the display and access of subcollection data. Thank you!

[rishipurwar1]

Rowy should be able to detect and suggest the schema for both subtables and array subtables

Hello @shamsmosowi I did some research on how we're fetching the data from firestore for each collection:

1. In TableSourceFirestore.js, we call the useFirestoreCollectionAtom hook to fetch the docs from firestore and save them to tableRowsDbAtom.
2. Then, we access the tableRowsDbAtom in the table.ts file, which contains all the documents for a particular collection.
3. Then we merge the two arrays after removing the duplicates and update the value of tableRowsAtom.
4. Finally, we access the tableRowsAtom in the EmptyTable component to show the user that you already have some data in the firestore.

Am I right up to this point?

Now, to retrieve data for subcollections, we can use the same approach as we do for collections since fetching data for collections (collection(firebaseDb, pathToCollection)) and subcollections (collection(firebaseDb, pathToSubCollection)) works in the same way. However, we need to provide a way for the user to input the names of the subcollections, as each document can have multiple subcollections.

Approaches:

1. Provide a new field to the user at the time of filling out the "Create table" form just below the "Collection name" field.
2. Add one more tab in the "import" modal component to enter the subcollection names.

Another consideration is how we should present the subcollection data. For example, should we generate separate table pages for each subcollection, and how will users access these specific subcollection tables?

And Could you please tell me what you mean by "subtables" and "array subtables"?

Please let me know if you have any further thoughts or if there are any other aspects we should consider regarding the display and access of subcollection data. Thank you!

I just got to know that we can import the sub-collections data using the sub-Table field type.

[shamsmosowi]

Rowy should be able to detect and suggest the schema for both subtables and array subtables

Hello @shamsmosowi I did some research on how we're fetching the data from firestore for each collection:

1. In TableSourceFirestore.js, we call the useFirestoreCollectionAtom hook to fetch the docs from firestore and save them to tableRowsDbAtom.
2. Then, we access the tableRowsDbAtom in the table.ts file, which contains all the documents for a particular collection.
3. Then we merge the two arrays after removing the duplicates and update the value of tableRowsAtom.
4. Finally, we access the tableRowsAtom in the EmptyTable component to show the user that you already have some data in the firestore.

Am I right up to this point?
Now, to retrieve data for subcollections, we can use the same approach as we do for collections since fetching data for collections (collection(firebaseDb, pathToCollection)) and subcollections (collection(firebaseDb, pathToSubCollection)) works in the same way. However, we need to provide a way for the user to input the names of the subcollections, as each document can have multiple subcollections.
Approaches:

1. Provide a new field to the user at the time of filling out the "Create table" form just below the "Collection name" field.
2. Add one more tab in the "import" modal component to enter the subcollection names.

Another consideration is how we should present the subcollection data. For example, should we generate separate table pages for each subcollection, and how will users access these specific subcollection tables? And Could you please tell me what you mean by "subtables" and "array subtables"? Please let me know if you have any further thoughts or if there are any other aspects we should consider regarding the display and access of subcollection data. Thank you!

I just got to know that we can import the sub-collections data using the sub-Table field type.

oh 😅 then mainly need the same implementation for array subtables think that one is not implement yet, you can double check

[rishipurwar1]

need the same implementation for array subtables think that one is not implement yet

Are you talking about this "array subtables" implementation?

Screen.Recording.2023-06-13.at.4.05.52.PM.mov

and I think this is the implementation for subtables:

Screen.Recording.2023-06-13.at.4.16.12.PM.mov

[harinij]

Hi @rishipurwar1 - adding more details below to make the task more clear.

Currently, when you add a subtable (subcollection) column and open it - if there is existing data in the subcollection - then there is an indicator that there is data already present and if you would like to import. - which is the expected behaviour.

However, when you add an array subtable column with existing data, then it says there is no data available and you have to manually click add columns. For this case, we would like for the automatic import option to be available with suggestion for field types.

Recorded a video for this to indicate how Subtable works as expected whereas Array subtable does not. https://jam.dev/c/7ef3bccd-656c-4d25-ba5e-49efc9e485c7

I think in the case of the video above, for the Array Subtable type, you potentially had added the columns earlier? which is why it shows up. If you try for a brand new array added to firestore and then add the array subtable column it will not work and will be like how it is in the video I recorded above in the second part.

Hope that clarifies things, let us know if any questions.

[rishipurwar1]

Hi @harinij The task is now much clearer to me. I just need to work on the array subtable part, not on the subtable(already implemented). Thanks for your clarification.