Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP :update the annotation sql file #196

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
190 changes: 190 additions & 0 deletions Annotation_V2/create_db.py
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When programming with a large group of people, it's absolutely vital to try to avoid repetitive code as much as possible. Here, you are basically making a new "copy" of the database but without any of the neat validation rules that the script above implements.

Also, all database related schemas have always been placed in Database/schema so there is no justification for a new directory.

These changes need to be implemented in Database/schema in a fashion similar to poetry run python3 Database/schema/create_db.py -db <DB-NAME> and with proper validation rules.

We now have two files called create_db.py. This kind of repetition is confusing and dangerous down the line.

Original file line number Diff line number Diff line change
@@ -0,0 +1,190 @@
import sqlite3

# Connect to SQLite database (or create it if it doesn't exist)
conn = sqlite3.connect("/home/nl/Wikimpacts/Annotation_V2/impactDB_gold_V2.db")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's not okay to have these paths hard-coded. You need to use the relative path where the repository is the root directory (so starting from Wikimpacts/).

cursor = conn.cursor()

# Create tables
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is redundant.

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Basic (
Event_ID TEXT PRIMARY KEY,
Main_Event TEXT,
Hazards TEXT,
Event_Names TEXT,
Sources TEXT,
Start_Date_Year INTEGER,
Start_Date_Month INTEGER,
Start_Date_Day INTEGER,
End_Date_Year INTEGER,
End_Date_Month INTEGER,
End_Date_Day INTEGER
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Affected_Countries (
Country TEXT,
Event_ID TEXT,
PRIMARY KEY (Event_ID, Country),
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Deaths (
Event_ID TEXT PRIMARY KEY,
Total_Deaths_Raw TEXT,
Total_Deaths_Min INTEGER,
Total_Deaths_Max INTEGER,
Total_Deaths_Approx BOOLEAN,
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Injuries (
Event_ID TEXT PRIMARY KEY,
Total_Injuries_Raw TEXT,
Total_Injuries_Min INTEGER,
Total_Injuries_Max INTEGER,
Total_Injuries_Approx BOOLEAN,
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Affected (
Event_ID TEXT PRIMARY KEY,
Total_Affected_Raw TEXT,
Total_Affected_Min INTEGER,
Total_Affected_Max INTEGER,
Total_Affected_Approx BOOLEAN,
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Homeless (
Event_ID TEXT PRIMARY KEY,
Total_Homeless_Raw TEXT,
Total_Homeless_Min INTEGER,
Total_Homeless_Max INTEGER,
Total_Homeless_Approx BOOLEAN,
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Displaced (
Event_ID TEXT PRIMARY KEY,
Total_Displaced_Raw TEXT,
Total_Displaced_Min INTEGER,
Total_Displaced_Max INTEGER,
Total_Displaced_Approx BOOLEAN,
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Buildings_Damaged (
Event_ID TEXT PRIMARY KEY,
Total_Buildings_Damaged_Raw TEXT,
Total_Buildings_Damaged_Min INTEGER,
Total_Buildings_Damaged_Max INTEGER,
Total_Buildings_Damaged_Approx BOOLEAN,
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Damage (
Event_ID TEXT,
Total_Damage_Raw TEXT,
Total_Damage_Min INTEGER,
Total_Damage_Max INTEGER,
Total_Damage_Approx BOOLEAN,
Total_Damage_Unit TEXT,
Total_Damage_Inflation_Adjusted BOOLEAN,
Total_Damage_Inflation_Adjusted_Year INTEGER,
PRIMARY KEY (Total_Damage_Unit, Total_Damage_Inflation_Adjusted, Total_Damage_Inflation_Adjusted_Year, Event_ID),
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

cursor.execute(
"""
CREATE TABLE IF NOT EXISTS Total_Insured_Damage (
Event_ID TEXT,
Total_Insured_Damage_Raw TEXT,
Total_Insured_Damage_Min INTEGER,
Total_Insured_Damage_Max INTEGER,
Total_Insured_Damage_Approx BOOLEAN,
Total_Insured_Damage_Unit TEXT,
Total_Insured_Damage_Inflation_Adjusted BOOLEAN,
Total_Insured_Damage_Inflation_Adjusted_Year INTEGER,
PRIMARY KEY (Event_ID, Total_Insured_Damage_Unit, Total_Insured_Damage_Inflation_Adjusted, Total_Insured_Damage_Inflation_Adjusted_Year),
FOREIGN KEY (Event_ID) REFERENCES Basic(Event_ID)
);
"""
)

# Create tables for Instance_Per_Administrative_Areas
tables = [
"Instance_Per_Administrative_Areas_Deaths",
"Instance_Per_Administrative_Areas_Injuries",
"Instance_Per_Administrative_Areas_Homeless",
"Instance_Per_Administrative_Areas_Displaced",
"Instance_Per_Administrative_Areas_Affected",
"Instance_Per_Administrative_Areas_Buildings_Damaged",
"Instance_Per_Administrative_Areas_Damage",
"Instance_Per_Administrative_Areas_Insured_Damage",
]

for table in tables:
cursor.execute(
f"""
CREATE TABLE IF NOT EXISTS {table} (
Event_ID TEXT,
Country TEXT NOT NULL,
Country_GID TEXT,
Administrative_Areas TEXT,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this supposed to be an array or TEXT?

Administrative_Areas_GID TEXT,
Hazards TEXT,
Start_Date_Year INTEGER,
Start_Date_Month INTEGER,
Start_Date_Day INTEGER,
End_Date_Year INTEGER,
End_Date_Month INTEGER,
End_Date_Day INTEGER,
Num_Raw TEXT,
Num_Min INTEGER,
Num_Max INTEGER,
Num_Approx BOOLEAN,
PRIMARY KEY (Event_ID),
UNIQUE (Event_ID, Country, Administrative_Areas),
FOREIGN KEY (Event_ID) REFERENCES Affected_Countries(Event_ID)
);
"""
)

# Commit changes and close the connection
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment is redundant.

conn.commit()
conn.close()
3 changes: 3 additions & 0 deletions Annotation_V2/impactDB_gold_V2.db
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The choice of placing this database in a new folder called "Annotation_V2" is unhelpful and will clutter the repository. Gold annotations have always landed in Database/gold, so there is no reason for them to have a new directory. You need to reconsider where you want to place this data.

Git LFS file not shown