adding canada population stats over age and gender

statvar_imports/canada_statistics_age_gender/CanadaStatisticsAgeGender/README.md

@@ -0,0 +1,55 @@
+# Canada Annual Population Estimates
+
+## Overview
+
+This dataset contains annual population estimates for Canada, its provinces, and territories, sourced from Statistics Canada. It provides a comprehensive demographic overview from 1971 to the present, broken down by age and gender.
+
+The dataset features:
+- **Geographic Coverage:** Canada (national), provinces, and territories.
+- **Temporal Coverage:** Annual estimates (July 1st) from 1971 to the most recent year.
+- **Granularity:**
+    - **Age:** Single years of age (0 to 100+) and age groups.
+    - **Gender:** Total, Men+, Women+.
+- **Metrics:** Population counts (Persons).
+
+This data is crucial for understanding demographic trends, population aging, and regional growth patterns across Canada.
+
+## Data Acquisition
+
+To download the latest version of this data or refresh the dataset:
+
+1. **Navigate to the source:** [Table 17-10-0005-01: Population estimates on July 1st, by age and sex](https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000501)
+
+2. **Selection Criteria:**
+    - **Geography:** Select "Canada" and all provinces/territories.
+    - **Age group:** Select "All ages", individual years (0 to 100+), and standard age groups.
+    - **Sex:** Select "Total - sex", "Males", "Females".
+    - **Reference period:** Select from 1971 to the latest available year.
+
+3. **Download:**
+    - Select "Download options".
+    - Choose "CSV (comma delimited)" to get the standard dataset file.
+    - Save the file as `canada_annual_population_age_gender_input.csv` in the `gs://unresolved_mcf/country/canada_statistics_age_gender/input_files/` directory.
+
+## Processing Instructions
+
+To process the Canada Statistics data and generate statistical variables, use the following command from the repository root (or adjust paths relative to your current working directory):
+
+```python ./data/tools/statvar_importer/stat_var_processor.py 
+  --input_data="gs://unresolved_mcf/country/canada_statistics_age_gender/input_files/*.csv"
+  --pv_map="./data/statvar_imports/canada_statistics/CanadaStatisticsAgeGender/canada_annual_population_age_gender_pvmap.csv" 
+  --config_file="./data/statvar_imports/canada_statistics/CanadaStatisticsAgeGender/canada_annual_population_age_gender_metadata.csv" 
+  --output_path="output/canada_annual_population_age_gender_output"
+```
+
+## Data Refresh & Maintenance
+
+When Statistics Canada releases new annual updates (typically in late September):
+
+1. **Execute the Data Acquisition steps** to get the latest CSV.
+
+2. **Verify Column Names:** Check if Statistics Canada has changed any column names or codes (e.g., updates to "Gender" terminology). Update `canada_annual_population_age_gender_pvmap.csv` if necessary.
+
+3. **Check for New Geographies:** Ensure no new administrative regions have been added or codes changed.
+
+4. **Execute the processing step** to generate the new output files.

statvar_imports/canada_statistics_age_gender/CanadaStatisticsAgeGender/canada_annual_population_age_gender_metadata.csv

@@ -0,0 +1,3 @@
+parameter,value
+output_columns,"observationDate,value,observationAbout,variableMeasured"
+header_rows,1

statvar_imports/canada_statistics_age_gender/CanadaStatisticsAgeGender/canada_annual_population_age_gender_pvmap.csv

@@ -0,0 +1,159 @@
+key,p1,v1,p2,v2,p3,v3
+VALUE,measuredProperty,count,populationType,Person,value,{Number}
+REF_DATE,observationDate,{Number},,,,
+Canada,observationAbout,country/CAN,,,,
+Newfoundland and Labrador,observationAbout,wikidataId/Q2003,,,,
+Prince Edward Island,observationAbout,wikidataId/Q1979,,,,
+Nova Scotia,observationAbout,wikidataId/Q1952,,,,
+New Brunswick,observationAbout,wikidataId/Q1965,,,,
+Quebec,observationAbout,wikidataId/Q176,,,,
+Ontario,observationAbout,wikidataId/Q1904,,,,
+Manitoba,observationAbout,wikidataId/Q1948,,,,
+Saskatchewan,observationAbout,wikidataId/Q1989,,,,
+Alberta,observationAbout,wikidataId/Q1951,,,,
+British Columbia,observationAbout,wikidataId/Q1974,,,,
+Yukon,observationAbout,wikidataId/Q2009,,,,
+Northwest Territories,observationAbout,wikidataId/Q2007,,,,
+Nunavut,observationAbout,wikidataId/Q2023,,,,
+Total - gender,gender,"",,,,
+Men+,gender,Male,,,,
+Women+,gender,Female,,,,
+All ages,age,"",,,,
+0 to 4 years,age,[0 4 Years],,,,
+0 years,age,Years0,,,,
+1 year,age,Years1,,,,
+2 years,age,Years2,,,,
+3 years,age,Years3,,,,
+4 years,age,Years4,,,,
+5 to 9 years,age,[5 9 Years],,,,
+5 years,age,Years5,,,,
+6 years,age,Years6,,,,
+7 years,age,Years7,,,,
+8 years,age,Years8,,,,
+9 years,age,Years9,,,,
+10 to 14 years,age,[10 14 Years],,,,
+10 years,age,Years10,,,,
+11 years,age,Years11,,,,
+12 years,age,Years12,,,,
+13 years,age,Years13,,,,
+14 years,age,Years14,,,,
+15 to 19 years,age,[15 19 Years],,,,
+15 years,age,Years15,,,,
+16 years,age,Years16,,,,
+17 years,age,Years17,,,,
+18 years,age,Years18,,,,
+19 years,age,Years19,,,,
+20 to 24 years,age,[20 24 Years],,,,
+20 years,age,Years20,,,,
+21 years,age,Years21,,,,
+22 years,age,Years22,,,,
+23 years,age,Years23,,,,
+24 years,age,Years24,,,,
+25 to 29 years,age,[25 29 Years],,,,
+25 years,age,Years25,,,,
+26 years,age,Years26,,,,
+27 years,age,Years27,,,,
+28 years,age,Years28,,,,
+29 years,age,Years29,,,,
+30 to 34 years,age,[30 34 Years],,,,
+30 years,age,Years30,,,,
+31 years,age,Years31,,,,
+32 years,age,Years32,,,,
+33 years,age,Years33,,,,
+34 years,age,Years34,,,,
+35 to 39 years,age,[35 39 Years],,,,
+35 years,age,Years35,,,,
+36 years,age,Years36,,,,
+37 years,age,Years37,,,,
+38 years,age,Years38,,,,
+39 years,age,Years39,,,,
+40 to 44 years,age,[40 44 Years],,,,
+40 years,age,Years40,,,,
+41 years,age,Years41,,,,
+42 years,age,Years42,,,,
+43 years,age,Years43,,,,
+44 years,age,Years44,,,,
+45 to 49 years,age,[45 49 Years],,,,
+45 years,age,Years45,,,,
+46 years,age,Years46,,,,
+47 years,age,Years47,,,,
+48 years,age,Years48,,,,
+49 years,age,Years49,,,,
+50 to 54 years,age,[50 54 Years],,,,
+50 years,age,Years50,,,,
+51 years,age,Years51,,,,
+52 years,age,Years52,,,,
+53 years,age,Years53,,,,
+54 years,age,Years54,,,,
+55 to 59 years,age,[55 59 Years],,,,
+55 years,age,Years55,,,,
+56 years,age,Years56,,,,
+57 years,age,Years57,,,,
+58 years,age,Years58,,,,
+59 years,age,Years59,,,,
+60 to 64 years,age,[60 64 Years],,,,
+60 years,age,Years60,,,,
+61 years,age,Years61,,,,
+62 years,age,Years62,,,,
+63 years,age,Years63,,,,
+64 years,age,Years64,,,,
+65 to 69 years,age,[65 69 Years],,,,
+65 years,age,Years65,,,,
+66 years,age,Years66,,,,
+67 years,age,Years67,,,,
+68 years,age,Years68,,,,
+69 years,age,Years69,,,,
+70 to 74 years,age,[70 74 Years],,,,
+70 years,age,Years70,,,,
+71 years,age,Years71,,,,
+72 years,age,Years72,,,,
+73 years,age,Years73,,,,
+74 years,age,Years74,,,,
+75 to 79 years,age,[75 79 Years],,,,
+75 years,age,Years75,,,,
+76 years,age,Years76,,,,
+77 years,age,Years77,,,,
+78 years,age,Years78,,,,
+79 years,age,Years79,,,,
+80 to 84 years,age,[80 84 Years],,,,
+80 years,age,Years80,,,,
+81 years,age,Years81,,,,
+82 years,age,Years82,,,,
+83 years,age,Years83,,,,
+84 years,age,Years84,,,,
+85 to 89 years,age,[85 89 Years],,,,
+85 years,age,Years85,,,,
+86 years,age,Years86,,,,
+87 years,age,Years87,,,,
+88 years,age,Years88,,,,
+89 years,age,Years89,,,,
+0 to 14 years,age,[0 14 Years],,,,
+0 to 15 years,age,[0 15 Years],,,,
+0 to 16 years,age,[0 16 Years],,,,
+0 to 17 years,age,[0 17 Years],,,,
+15 to 49 years,age,[15 49 Years],,,,
+15 to 64 years,age,[15 64 Years],,,,
+16 to 64 years,age,[16 64 Years],,,,
+17 to 64 years,age,[17 64 Years],,,,
+18 years and older,age,[18 - Years],,,,
+18 to 64 years,age,[18 64 Years],,,,
+18 to 24 years,age,[18 24 Years],,,,
+25 to 44 years,age,[25 44 Years],,,,
+45 to 64 years,age,[45 64 Years],,,,
+65 years and older,age,[65 - Years],,,,
+90 years and older,age,[90 - Years],,,,
+Median age,#ignore,ignore,,,,
+Average age,#ignore,ignore,,,,
+90 to 94 years,age,[90 94 Years],,,,
+90 years,age,Years90,,,,
+91 years,age,Years91,,,,
+92 years,age,Years92,,,,
+93 years,age,Years93,,,,
+94 years,age,Years94,,,,
+95 to 99 years,age,[95 99 Years],,,,
+95 years,age,Years95,,,,
+96 years,age,Years96,,,,
+97 years,age,Years97,,,,
+98 years,age,Years98,,,,
+99 years,age,Years99,,,,
+100 years and older,age,[100 - Years],,,,

statvar_imports/canada_statistics_age_gender/CanadaStatisticsAgeGender/manifest.json

@@ -0,0 +1,25 @@
+{
+    "import_specifications": [
+        {
+            "import_name": "CanadaStatisticsAgeGender",
+            "curator_emails": [
+                "support@datacommons.org"
+            ],
+            "provenance_url": "https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1710000501",
+            "provenance_description": "Canada Population Yearly Esitmates at Country, Province and Territories level across different age groups and gender.",
+            "scripts": [
+                "../../tools/statvar_importer/stat_var_processor.py --input_data=gs://unresolved_mcf/country/canada_statistics_age_gender/input_files/*.csv --pv_map=canada_annual_population_age_gender_pvmap.csv --config_file=canada_annual_population_age_gender_metadata.csv --output_path=output/canada_annual_population_age_gender_output --existing_statvar_mcf=gs://unresolved_mcf/scripts/statvar/stat_vars.mcf"
+            ],
+            "import_inputs": [
+                {
+                    "template_mcf": "output/canada_annual_population_age_gender_output.tmcf",
+                    "cleaned_csv": "output/canada_annual_population_age_gender_output.csv"
+                }
+            ],
+            "source_files": [
+                "gs://unresolved_mcf/country/canada_statistics_age_gender/input_files/*.csv"
+            ],
+            "cron_schedule": "30 5 1 *  *"
+        }
+    ]
+}