Tidying up licences and unused files and code

• Cleanup of unused files and features #562, #561
• Removes unused APIs #560
• Improves log level to reduce logging output #559
• Fix the handling of licences during import and indexing #558

Tools for single-dataset updating.

• Adds replication script for obtaining a DB dump
• Adds rake task to import a single dataset (and reindex it)
• Adds access_constraints field to InspireMetadata
• Updates Rails version
• Provides more RAM to the worker in production

Upgrade Loofah

Upgrade Loofah #543

English mappings. Improvements to importing, logging & deployments. Ruby 2.5.

• Add English field mappings to datasets index #534
• Improvements to importing #535
• Improvements to logging #536
• Improvements to deployment scripts #538
• Remove short id logic #533
• Final cleanup of themes #531
• Upgrade to Ruby 2.5.0 #541 #542

Config edits, infra updates & clean up

• Remove Themes (now replaced by Topics)
• Upgrade deprecated Elasticsearch string types
• Update elasticsearch config in initialiser
• Replace Blue-Green-Deploy Cloud Foundry plugin with Autopilot

Upgrade Rails to 5.1.5 from 5.1.4

Rails 5.1.5 has been released

http://weblog.rubyonrails.org/2018/2/14/Rails-5-1-5-has-been-released/

Bug fix for indexer

• Fixes undefined variable index_name in DatasetIndexerService

Import and reindex tasks synchronously

• Run import and reindex tasks synchronously until we find a fix for the Redis timeouts

Topics, dashboard, link checker tests, dataset/datafile short ids

New

• topics for datasets
• dashboard to show what is currently in the database (dataset count, datafile count, broken links, etc.)
• short ids for Datasets and Links
• write tests for the link checker + refactor code
• more explicit staging/prod database config parity

Bug fixes

• Make sure three indices are retained when reindexing datasets

Bug fix

• Organisation import fix