Skip to content

Clone of #5852#5984

Open
beets wants to merge 42 commits intodatacommonsorg:masterfrom
beets:5852-fork
Open

Clone of #5852#5984
beets wants to merge 42 commits intodatacommonsorg:masterfrom
beets:5852-fork

Conversation

@beets
Copy link
Collaborator

@beets beets commented Feb 10, 2026

Needed to get it up to HEAD. See that PR for context on changes.

nick-nlb and others added 30 commits February 9, 2026 20:26
…ocation in homepage, because it is now used in multiple areas).
…(in particular, implementing functionality that allows us to display a version used by the coming custom home page).
… that is no longer used, and fulfilling a TODO).
…der.json. If we are in a custom DC, we will look in a particular directory (config/custom_dc) and, if we find a `header.json` in the enclosed custom directory, we will use that.

With this, we will be able to use the standard Data Commons header, while along custom DCs to customize the actual header content.
…conBox component that allows us to display descriptions as used on the new custom home page view.
… app, which will display the search bar and the tool list.
…ws it to be displayed anywhere on the site as a regular content component item.
…and display both the main site's header (with the custom content) as well as the new custom home page content (styling still coming).
… than HEADER_MENU_V2 (which has now been removed).

We also explicitly add the tool list because we have now added a stat var explorer tool to the list which isn't to display on the home page (by default we display all the tool types if none are given).
Clean up HTML and Classes from the main wrappers and components, keeping the same logic and functionality
• Show Load more now its handled in the beginning
• The results can be infinite the wrapper nows scrolls past the visible threshold
nick-nlb and others added 6 commits February 9, 2026 20:28
With some of the class names having changed, the tests are now updated to compensate. Additionally, the test that counts the number of search results now only counts the actual search results (which number 4 because, in order to properly space the elements in the search results, we lowered the number to display from 5 to 4 to fit the show more equally). This means that we are checking for 4 search results, instead of the previous 6.
This reverts commit 34d5d48.
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @beets, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on a significant modernization of the frontend architecture, particularly for custom Data Commons deployments. It streamlines the configuration of UI elements like headers and footers, introduces a more modular component-based approach for the homepage and search functionality, and updates underlying dependencies. These changes aim to enhance customization capabilities, improve code maintainability, and prepare the application for future feature development.

Highlights

  • Submodule Updates: Updated the import and mixer submodules to their latest commits, bringing in external dependency changes.
  • Custom Configuration Overrides: Implemented logic in server/__init__.py to support custom header and footer JSON configurations, allowing custom Data Commons instances to override default menu structures.
  • Frontend Component Refactoring: Introduced new reusable React components for Search and Tools (static/js/components/content/search.tsx, static/js/components/content/tools.tsx), centralizing common UI elements and improving modularity.
  • Homepage Modernization: Migrated the custom Data Commons homepage to a new React application structure (static/js/apps/homepage/custom_dc_app.tsx), replacing static HTML content with dynamic, component-based rendering.
  • Search Bar Enhancements: Refactored the natural language search bar (auto_complete_input.tsx, auto_complete_suggestions.tsx) using Emotion styled components for improved styling flexibility and consistency, and updated autocomplete result handling.
  • Header/Footer Structure Update: Modified server/config/base/header.json to a more structured format with id, introduction, and primarySectionGroups, and streamlined the custom header configuration by renaming header_v2.json and removing certain sections.
  • WebDriver Test Adjustments: Updated WebDriver tests to reflect changes in DOM structure, specifically by using data-testid attributes and CSS_SELECTOR for element identification, and adjusted expected autocomplete result counts.
Changelog
  • import
    • Updated submodule commit from 863c814 to 8595ec5.
  • mixer
    • Updated submodule commit from 4a70b4b to ee1898d.
  • server/init.py
    • Added get_custom_config_path function to retrieve custom configuration folder paths.
    • Modified inject_common_parameters to support dynamic loading and overriding of header.json and footer.json based on custom configurations.
    • Removed references to HEADER_MENU_V2 and header_v2.json.
  • server/app_env/custom.py
    • Updated LOGO_PATH to point to a .svg file instead of a .png file.
  • server/config/base/header.json
    • Restructured menu items to include id, introduction, and primarySectionGroups for better organization and detailed descriptions.
    • Added a new 'Feedback' menu item.
  • server/config/base/header_v2.json
    • Renamed to server/config/custom_dc/custom/base/header.json.
    • Removed 'About' and 'Feedback' sections from the custom header configuration.
  • server/templates/base.html
    • Changed data-header attribute to use HEADER_MENU instead of HEADER_MENU_V2.
  • server/templates/custom_dc/custom/base.html
    • Updated stylesheet link from static.min.css to core.min.css.
    • Added base.js script for core functionality.
    • Expanded metadata-base div to include additional data attributes for configuration.
    • Removed hardcoded nav and footer HTML elements, shifting rendering responsibility to client-side JavaScript.
  • server/templates/custom_dc/custom/homepage.html
    • Updated copyright year to 2025.
    • Added an external_icon macro for SVG icon rendering.
    • Included homepage.min.css and homepage_custom_dc.js for homepage-specific styling and scripting.
    • Replaced static 'Home Page' content with an empty app-container div for React application rendering.
  • server/webdriver/cdc_tests/nl_test.py
    • Updated CSS selector for the search bar from .search-bar to .search-section.
  • server/webdriver/tests/homepage_test.py
    • Modified element location strategy from By.CLASS_NAME to By.CSS_SELECTOR using data-testid for search input result sections.
    • Adjusted the expected count of initial autocomplete results for statistical variables from 6 to 4.
  • static/custom_dc/custom/logo.svg
    • Added a new SVG file for the custom Data Commons logo.
  • static/js/apps/explore/search_section.tsx
    • Wrapped the NlSearchBar component within a div with className="search-section".
  • static/js/apps/homepage/app.tsx
    • Removed local import of the Tools component.
    • Imported the Tools component from the shared ../../components/content/tools path.
    • Updated the usage of the Tools component to pass tools array and children props.
  • static/js/apps/homepage/components/tools.tsx
    • Removed the file, as its functionality was moved to a shared component.
  • static/js/apps/homepage/custom_dc_app.tsx
    • Added a new React component (App) for the custom Data Commons homepage, integrating Search and Tools components.
  • static/js/apps/homepage/main_custom_dc.ts
    • Modified to load locale data and render the new custom_dc_app into the app-container div on page load.
  • static/js/components/content/search.tsx
    • Added a new React component for a reusable search bar, supporting customizable introductory content.
  • static/js/components/content/tools.tsx
    • Added a new React component for a reusable tools section, allowing configuration of displayed tools and their descriptions.
  • static/js/components/elements/button/button.tsx
    • Added a new naked variant to the Button component, providing a style without border, background, or padding.
  • static/js/components/elements/icons/location.tsx
    • Added a new React component for a Material Design 'Location City' SVG icon.
  • static/js/components/elements/link_icon_box.tsx
    • Added an optional description prop to LinkIconBox.
    • Updated rendering to display the link title and an optional description separately.
  • static/js/components/nl_search_bar/auto_complete_input.tsx
    • Added /** @jsxImportSource @emotion/react */ pragma for Emotion styling.
    • Introduced SearchWrapper and SearchButton styled components for the search bar's visual elements.
    • Defined searchInputCss for consistent input field styling.
    • Replaced existing HTML structure with styled components for the search input and button.
    • Updated search icon rendering.
  • static/js/components/nl_search_bar/auto_complete_suggestions.tsx
    • Added /** @jsxImportSource @emotion/react */ pragma for Emotion styling.
    • Introduced AutoCompleteSuggestionsWrapper, SuggestionRow, and LoadMoreWrapper styled components for autocomplete suggestions.
    • Replaced the material-icons-outlined location icon with the new Location SVG component.
    • Adjusted the logic for INITIAL_VISIBLE_RESULTS based on the presence of statistical variable search results.
    • Updated the 'Load More' text in the suggestions.
  • static/js/components/nl_search_bar/nl_search_bar_header_inline.tsx
    • Added /** @jsxImportSource @emotion/react */ pragma for Emotion styling.
  • static/js/components/nl_search_bar/nl_search_bar_standard.tsx
    • Removed extraneous wrapping div elements, making AutoCompleteInput the direct return of the component.
  • static/js/theme/theme.ts
    • Added x2s (2 pixels) to the SPACING constants.
    • Defined new search and searchSuggestions theme properties, including colors, dimensions, and states for search UI elements.
  • static/js/theme/types.ts
    • Added x2s to the spacing interface.
    • Added search and searchSuggestions interfaces to the Theme type, reflecting the new theme properties.
Activity
  • This pull request is a clone of an existing PR (Custom dc homepage #5852), indicating that the changes were previously developed and are now being rebased and brought up to date with the current codebase's HEAD.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant and well-executed refactoring of the frontend, migrating the header, footer, homepage, and search components to a more modern, reusable, and theme-based architecture using Emotion. This is a great step forward for maintainability. The backend changes support custom configurations, and tests have been updated accordingly. My review focuses on a key performance improvement opportunity in the backend, along with a minor code cleanup suggestion to further enhance code quality.

Comment on lines +515 to +525
custom_config_path = get_custom_config_path(custom_dc_template_folder)
if custom_config_path:
custom_header_override = os.path.join(custom_config_path, "base",
"header.json")
if os.path.exists(custom_header_override):
header_json_path = custom_header_override

custom_footer_override = os.path.join(custom_config_path, "base",
"footer.json")
if os.path.exists(custom_footer_override):
footer_json_path = custom_footer_override
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The logic to determine custom header and footer paths, including os.path.exists checks, is inside the inject_common_parameters context processor. This means it runs on every request, causing unnecessary file system access and impacting performance. This logic should be moved to the create_app function to be executed only once at application startup. The resulting paths can then be stored in app.config and accessed from there in the context processor.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a todo and we can fix this in a follow-up

Comment on lines +500 to +503
"""
Return the custom configuration folder path
if custom configuration is enabled and a custom template folder is provided.
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The docstring for get_custom_config_path is not correctly indented according to PEP 257. The content of a multiline docstring should be indented to the same level as the opening quotes. I've also taken the liberty to improve the wording for clarity and adopt a more standard format.

    """Gets the path for a custom configuration folder.

    Returns:
        The path to the custom configuration folder if custom configuration is
        enabled and a folder is provided, otherwise None.
    """

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix

@dwnoble dwnoble self-requested a review February 11, 2026 05:21
Copy link
Contributor

@dwnoble dwnoble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @beets !

I successfully tested this with:

DC_API_KEY=<my-api-key> ./run_server.sh  -m -e custom 

And verified other custom environments work like:

DC_API_KEY=<my-api-key> ./run_server.sh  -m -e stanford

lgtm overall, can you just:

  • Fix indentation warning raised by gemini
  • Merge in the latest master branch
  • Reset the mixer and import submodules to match the version on master

Comment on lines +500 to +503
"""
Return the custom configuration folder path
if custom configuration is enabled and a custom template folder is provided.
"""
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix

Comment on lines +515 to +525
custom_config_path = get_custom_config_path(custom_dc_template_folder)
if custom_config_path:
custom_header_override = os.path.join(custom_config_path, "base",
"header.json")
if os.path.exists(custom_header_override):
header_json_path = custom_header_override

custom_footer_override = os.path.join(custom_config_path, "base",
"footer.json")
if os.path.exists(custom_footer_override):
footer_json_path = custom_footer_override
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a todo and we can fix this in a follow-up

@beets
Copy link
Collaborator Author

beets commented Feb 24, 2026

thanks for the review! i've made the changes, ptal

@beets
Copy link
Collaborator Author

beets commented Feb 25, 2026

i also applied the change to avoid looking for config files on every request. tested as well with the commands you shared and things are looking good. ptal

@nick-nlb nick-nlb self-requested a review February 26, 2026 19:36
Copy link
Contributor

@nick-nlb nick-nlb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you again for doing the rebase and bringing this branch up-to-date!

It's looking and testing well. It just needs a run_test.sh -f to apply formatting changes on the python side before merging in.

We discussed this offline, but just to remind us here, we will wait until after we meet and coordinate with @kmoscoe on before doing the actual merge.

Copy link
Contributor

@kmoscoe kmoscoe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could I make a request? Could we remove the other files that are no longer needed in server/templates/custom_dc/custom/ , namely about.html, faq.html and disclaimers.html. Now that we've removed the links to them from the home page, it's just confusing and cluttering to have them hanging around.


{% block head %}
<link rel="stylesheet" href={{url_for('static', filename='css/homepage.min.css', t=config['VERSION'])}}>
<script src={{url_for('static', filename='homepage_custom_dc.js', t=config['VERSION'])}} async></script>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this file actually coming from, and what is it used for? I couldn't find anything in static with this filename. Is it supposed to be js/apps/homepage/main_custom_dc.ts‎? Or does it only get compiled at run time? (I couldn't find it stored locally after starting up the server locally either.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants