Error: "no such table: solid_queue_recurring_tasks"

Greetings!

I'm all fired up after Rails World, and I'm getting started with a new Rails 8 application. I've installed this gem, along with mission_control-jobs. My stack is otherwise just a brand new Rails 8 application using SQLite.

I'm seeing the following error when trying to access the "Recurring" tab for Mission Control:

SQLite3::SQLException: no such table: solid_queue_recurring_tasks

I can see the table listed in db/queue_schema.rb, but it's not listed in the tables of my development database either. I'm not really sure what's going on beyond these simple facts.

@jherdman

Hey @jherdman, thanks for trying this! Rails World was epic 🌍 💎 🙇‍♀️

Just to be sure, did you run bin/rails db:prepare after solid_queue:install? What version of Solid Queue are you using?

Hi Rosa.

Yup! More or less...

rails new my_app
rails solid_queue:install
Ensure SQ is set up as my queue adapter
rails db:create db:migrate
bin/jobs 💥 fails with the error noted above 💥

It's super weird: all of the other tables are listed except recurring tasks.

That's super odd indeed, especially in a new app. I was going to ask whether you were using Solid Queue before, because if you were, that could certainly be the problem, but this is a new Rails app. Could you try resetting the DB? With bin/rails db:reset.

No luck :(

What version of Solid Queue are you using?

1.0

I've tried to reproduce following the same steps with a new app using Rails 8.0.0.beta1 but I can't, all the tables get created just fine as expected. First, are you running this in the development environment? At first, I thought it was production but seeing what you run, it seems it's development. Solid Queue is not configured by default in development so that could be the issue. Could you copy the contents of your development configuration in database.yml? Are you also setting

config.solid_queue.connects_to = { database: { writing: :queue } }

in config/environments/development.rb?

Yes, definitely a development environment. I'm using devcontainers if that's at all relevant?

I copied over the configuration you mentioned to my config/environments/development.rb file and recreated my DB. No luck. Here's the relevant files you've asked for:

# config/environments/development.rb
# frozen_string_literal: true

require 'active_support/core_ext/integer/time'

Rails.application.configure do
  # Settings specified here will take precedence over those in config/application.rb.

  # Make code changes take effect immediately without server restart.
  config.enable_reloading = true

  # Do not eager load code on boot.
  config.eager_load = false

  # Show full error reports.
  config.consider_all_requests_local = true

  # Enable server timing.
  config.server_timing = true

  # Enable/disable Action Controller caching. By default Action Controller caching is disabled.
  # Run rails dev:cache to toggle Action Controller caching.
  if Rails.root.join('tmp/caching-dev.txt').exist?
    config.action_controller.perform_caching = true
    config.action_controller.enable_fragment_cache_logging = true
    config.public_file_server.headers = { 'cache-control' => "public, max-age=#{2.days.to_i}" }
  else
    config.action_controller.perform_caching = false
  end

  # Change to :null_store to avoid any caching.
  config.cache_store = :memory_store

  # Store uploaded files on the local file system (see config/storage.yml for options).
  config.active_storage.service = :local

  # Don't care if the mailer can't send.
  config.action_mailer.raise_delivery_errors = false

  # Make template changes take effect immediately.
  config.action_mailer.perform_caching = false

  # Set localhost to be used by links generated in mailer templates.
  config.action_mailer.default_url_options = { host: 'localhost', port: 3000 }

  # Print deprecation notices to the Rails logger.
  config.active_support.deprecation = :log

  # Raise an error on page load if there are pending migrations.
  config.active_record.migration_error = :page_load

  # Highlight code that triggered database queries in logs.
  config.active_record.verbose_query_logs = true

  # Append comments with runtime information tags to SQL queries in logs.
  config.active_record.query_log_tags_enabled = true

  # Highlight code that enqueued background job in logs.
  config.active_job.verbose_enqueue_logs = true
  config.active_job.queue_adapter = :solid_queue
  config.solid_queue.connects_to = { database: { writing: :queue } }

  # Raises error for missing translations.
  # config.i18n.raise_on_missing_translations = true

  # Annotate rendered view with file names.
  config.action_view.annotate_rendered_view_with_filenames = true

  # Uncomment if you wish to allow Action Cable access from any origin.
  # config.action_cable.disable_request_forgery_protection = true

  # Raise error when a before_action's only/except options reference missing actions.
  config.action_controller.raise_on_missing_callback_actions = true

  # Apply autocorrection by RuboCop to files generated by `bin/rails generate`.
  config.generators.apply_rubocop_autocorrect_after_generate!

  ## LOOKBOOK

  config.view_component.default_preview_layout = 'component_preview'

  config.lookbook.preview_display_options = {
    theme: %w[light dark] # dynamic 'theme' display option
  }
end

# database.yml

# SQLite. Versions 3.8.0 and up are supported.
#   gem install sqlite3
#
#   Ensure the SQLite 3 gem is defined in your Gemfile
#   gem "sqlite3"
#
default: &default
  adapter: sqlite3
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  <<: *default
  database: storage/development.sqlite3

# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
  <<: *default
  database: storage/test.sqlite3


# Store production database in the storage/ directory, which by default
# is mounted as a persistent Docker volume in config/deploy.yml.
production:
  primary:
    <<: *default
    database: storage/production.sqlite3
  cache:
    <<: *default
    database: storage/production_cache.sqlite3
    migrations_paths: db/cache_migrate
  queue:
    <<: *default
    database: storage/production_queue.sqlite3
    migrations_paths: db/queue_migrate
  cable:
    <<: *default
    database: storage/production_cable.sqlite3
    migrations_paths: db/cable_migrate

Ahhh! The issue is that the DB configuration is missing in development, here:

# database.yml

# ...

development:
  <<: *default
  database: storage/development.sqlite3

Could you change this to be:

development:
  primary:
    <<: *default
    database: storage/development.sqlite3
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate

Hmmm... I think we may have bumped into a Rails bug:

vscode ➜ /workspaces/my_app (main) $ rails db:prepare

bin/rails aborted!
ArgumentError: No database file specified. Missing argument: database (ArgumentError)

          raise ArgumentError, "No database file specified. Missing argument: database"
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Tasks: TOP => db:prepare
(See full trace by running task with --trace)

Oh, that could be wrong indenting in the yaml file if you copied what I wrote above, as I wrote it directly in GitHub's comment box, so the indentation might be off 😅

Totally a copy/paste error on my part!

Pasting to help any future travellers:

development:
  <<: *default # <<< THIS IS MY MISTAKE! DELETE THIS LINE
  primary:
    <<: *default
    database: storage/development.sqlite3
  queue:
    <<: *default
    database: storage/development_queue.sqlite3
    migrations_paths: db/queue_migrate

OK, so the big take away (and surprise) for me was that SQ wasn't configured for development out of the box. This is a bit surprising as other queue systems (e.g. Good Job, Sidekiq) work in development out of the box.

If you're amenable we could solve for the surprise with documentation?

Got it!

OK, so the big take away (and surprise) for me was that SQ wasn't configured for development out of the box. This is a bit surprising as other queue systems (e.g. Good Job, Sidekiq) work in development out of the box.

Rails doesn't configure solid_queue in development because it's intended to be used in production, and the async adapter in development. There's a small note about this in the Installation section in the README, but very small 😅 Improving this is already recorded in #332, I think that would cover this.

Yeah, agreed re #332 . I'm going to close this as resolved.

Many thanks for the assist!

That's super odd indeed, especially in a new app. I was going to ask whether you were using Solid Queue before, because if you were, that could certainly be the problem, but this is a new Rails app. Could you try resetting the DB? With bin/rails db:reset.

Hey rosa! I'm encountering a similar issue. db/queue_schema.rb is empty as well. Although, I can see that rails db:reset does create the queue database. But running rail db:migrate essentially does nothing for queue schema.

But running rail db:migrate essentially does nothing for queue schema.

Yes, that's expected! db:migrate would run pending migrations, but there would be none for the queue DB.

But running rail db:migrate essentially does nothing for queue schema.

Yes, that's expected! db:migrate would run pending migrations, but there would be none for the queue DB.

Makes sense! Strange thing though is that rails db:prepare does nothing either.

Huh, that should work, db:prepare should create your queue DB if it doesn't exist, and load the schema. Maybe this is because your DB already exists and is loaded, or is it because your db/queue_schema.rb file is empty?

Huh, that should work, db:prepare should create your queue DB if it doesn't exist, and load the schema. Maybe this is because your DB already exists and is loaded, or is it because your db/queue_schema.rb file is empty?

Ahh, you're right. Somehow, db/queue_schema.rb gets wiped sometimes.

@gczh

@gczh, that's a Rails bug that was already fixed: rails/rails#52829

Rails doesn't configure solid_queue in development because it's intended to be used in production, and the async adapter in development.

Personally I think this is a really odd choice. As soon as your Rails app uses any database adapter, using solid_queue in development comes basically for free. And I would bet that every development team strifes to use a setup as similar to production in development, so I really doubt the async adapter is actually used in many projects.

Maybe it makes the initial setup a bit simpler. But it seems to have the potential to cause a bit of confusion shortly after.

@rosa

@rosa +1 @mediafinger, solid_queue might just be enabled in development as well and if anyone wants to test things async they can just use perform_now.

I've been bitten by this too, trying to run solid queue in development with just (although I see now it's in the docs):

  config.active_job.queue_adapter = :solid_queue

After adding config.solid_queue.connects_to and adding the queue in database.yml as it's done for production, it started working.

Rails doesn't configure solid_queue in development because it's intended to be used in production, and the async adapter in development.

Personally I think this is a really odd choice. As soon as your Rails app uses any database adapter, using solid_queue in development comes basically for free. And I would bet that every development team strifes to use a setup as similar to production in development, so I really doubt the async adapter is actually used in many projects.

Maybe it makes the initial setup a bit simpler. But it seems to have the potential to cause a bit of confusion shortly after.

I think the same. If I want to run solid queue in production, I will at least want to try it locally. Since it's so easy to setup, for me this would be a better default.

This has failed for me every time I've set up solid queue on any project. I simply don't understand how this can be the correct behavior. Rails doesn't use multiple db/migrate directories or db/schema.rb files for different environments, there is only this one set of DB definitions to describe all environments the app will run under.

It doesn't make any sense to me that because I do my development in the development environment, I would not get any DB definition files for the tables that are needed for the application to run successfully in the production environment. How is it intended that these tables ever actually get created?

This has failed for me every time I've set up solid queue on any project. I simply don't understand how this can be the correct behavior. Rails doesn't use multiple db/migrate directories or db/schema.rb files for different environments, there is only this one set of DB definitions to describe all environments the app will run under.

It doesn't make any sense to me that because I do my development in the development environment, I would not get any DB definition files for the tables that are needed for the application to run successfully in the production environment. How is it intended that these tables ever actually get created?

Same here, just set it up on a new project and have encountered the same issue again. I'm switching to GoodJob for now.

@tobico

@tobico, @gczh, if you want to use Solid Queue in development, you need to configure it for development, which is not done by default. You have the instructions here.

because I do my development in the development environment, I would not get any DB definition files for the tables that are needed for the application to run successfully in the production environment. How is it intended that these tables ever actually get created?

I don't understand what this means. The tables would be created in production when you run db:prepare.

@rosa

@rosa Solid Queue should follow the "principle of least surprise" and keep the configuration consistent over all environments. Given that all Rails developers are used to start the application with bin/dev the jobs server can be spun up automagically.

Keep the async option optional for those people that for some reason don't want to have the jobs running on a separate process.

This would actually lower the barrier from bringing the development app to production. And let you do the debugging locally the same way as later on production.

@mediafinger

@mediafinger I can see that, but it was decided that the async adapter was preferred for development by default. This is a decision made by Rails, which might change in the future, but for now, the configuration added on installation is just production. You can still configure Solid Queue for development following the instructions in the README; it's just a couple of things.

The original issue here is about the DB not being created, or trouble around loading the schema, no?

Navigation Menu

Repository navigation

Error: "no such table: solid_queue_recurring_tasks" #366

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Error: "no such table: solid_queue_recurring_tasks" #366

Description

Activity

rosa commented on Sep 29, 2024

jherdman commented on Sep 30, 2024

rosa commented on Sep 30, 2024

jherdman commented on Sep 30, 2024

rosa commented on Sep 30, 2024

jherdman commented on Sep 30, 2024

rosa commented on Sep 30, 2024

jherdman commented on Sep 30, 2024

rosa commented on Sep 30, 2024

jherdman commented on Sep 30, 2024

rosa commented on Sep 30, 2024

jherdman commented on Sep 30, 2024

rosa commented on Sep 30, 2024

jherdman commented on Sep 30, 2024

gczh commented on Oct 16, 2024

rosa commented on Oct 16, 2024

gczh commented on Oct 16, 2024

rosa commented on Oct 16, 2024

gczh commented on Oct 16, 2024

rosa commented on Oct 16, 2024

mediafinger commented on Oct 17, 2024

uurcank commented on Oct 17, 2024

dgmora commented on Dec 6, 2024

tobico commented on May 7, 2025

gczh commented on May 7, 2025

rosa commented on May 7, 2025

mediafinger commented on May 7, 2025

rosa commented on May 7, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions