Bitbucket

#Overview All the data for this project is stored in a bitbucket repository. It can be accessed through https://bitbucket.org/flipside-org/incendios-data

#if-adminstats

adminstats.json

This file contains data relative to each administrative area. Contains the amount of fires per year and their type as well as the amount of burnt area. It also contains a top section with the most significative event of that area.

Notes

This file was generated by using the script adminstats.js on a mongo collection created from the import of if-detailed/ifdata_detailed-04.csv

Import

First you need to import the data as it is:

$ mongoimport -d incendios -c statsadminareas < adminstats-X.json

Then you need to run a clean up script that will optimise it for the use in the application:

$ mongo < adminstats_cleanup.js

Run the round script to round the aa_total to two decimals:

$ mongo < adminstats_round.js

Example object

{
  "_id" : ObjectId("5177acf603f671df33a419db"),
  "aaid" : 8,
  "total" : 5537,
  "aa_total" : 111642.75,
  "top" : {
    "incendio" : {
      "date" : "2003-09-11",
      "aa_total" : 17213
    },
    "year" : 2003
  },
  "data" : [
    {
      "total" : 215,
      "nulls" : 0,
      "aa_total" : 3293.27,
      "incendio" : 54,
      "fogacho" : 138,
      "falso_alarme" : 0,
      "queimada" : 0,
      "agricola" : 23,
      "year" : 2001
    },
    {
      "total" : 274,
      "nulls" : 0,
      "aa_total" : 1833.14,
      "incendio" : 73,
      "fogacho" : 175,
      "falso_alarme" : 0,
      "queimada" : 0,
      "agricola" : 26,
      "year" : 2002
    },
    ...
  ]
}

---

if-detailed

Contains the data relative to the occurrences.

Parsing script

Mongo isn't able to import a file the detailed occurrences because it's one single geoJSON file.
This parsing file converts the detailed data from one giant geoJSON object to several smaller object.

Run the script

The script will parse the file ifdata_detailed-05.json and output ifdata_detailed-05_parsed.json

php ifdata_detailed-05_parsing_script.php

ORIGINAL:

{
"type": "FeatureCollection",
"features": [
  {feature...},
  {feature...},
  {feature...}
]
}

AFTER PARSING:

  {feature...}
  {feature...}
  {feature...}

---

if-general-stats

Occurences per year

DEPRECATED The detailed collection structure changed. The script needs to be updated.

The file occurrences_per_year.js generates output for the charts of the main page. It uses the detailed collection.

Run the script

Don't use the script if the json is already rendered.

mongo < occurrences_per_year.js

The script will create a collection named yearstatus.

Import the data.

mongoimport -d incendios -c yearstats < occurrences_per_year.json

Totals

DEPRECATED The detailed collection structure changed. The script needs to be updated.

The totals file outputs values for:

• Total occurrences
• Avg occurrences per year
• Total area burned
• Avg area burned per year

These values are printed in the shell and are meant to be copied.

mongo --quiet < totals.js

Biggest fires

DEPRECATED The detailed collection structure changed. The script needs to be updated.

This file only contains the code that is going to be used. For it to work the detailed collection needs an index on the aa_total.

Worst and least hit Concelho

This file only contains the code that is going to be used. Gets the data form the statsadminarea collections

---

if-mongo-exports

Contains the exports form the mongo database in a folder with the export date.

---

pt-administrative-areas

Stuff related with the administrative areas.
How to process the shape files: README.md

pt-admin-areas-2012-refactor

Refactor the objects from a plain object to a structured one.

Data

Import the data. (to be done from this folder)

mongoimport -d incendios -c geoadminareas --headerline --type csv < ../pt_admin_areas_2012-02.csv

pt-admin-areas-2012-refactor.js

Assumes:

• db : incendios
• col : geoadminareas

mongo < pt-admin-areas-2012-refactor.js

Refactors the data to match the needed structure:

Before

{
  "_id" : ObjectId("5190d0848ffb64ae45ded69f"),
  "aaid" : 1,
  "type" : 1,
  "parent_id" : 0,
  "freguesia" : "",
  "municipio" : "",
  "distrito" : "Aveiro",
  "area_t_ha" : "",
  "minx" : -8.1189363148,
  "miny" : 41.0134444112,
  "maxx" : -8.0599770395,
  "maxy" : 41.1008195317,
  "cntx" : -8.0894566771,
  "cnty" : 41.0571319714
}

After

{
  "_id" : ObjectId("5190d0848ffb64ae45ded69f"),
  "aaid" : 1,
  "type" : 1,
  "parent_id" : 0,
  "freguesia" : "",
  "municipio" : "",
  "distrito" : "Aveiro",
  "geo" : {
    "area" : "",
    "min" : {
      "x" : -8.1189363148,
      "y" : 41.0134444112
    },
    "max" : {
      "x" : -8.0599770395,
      "y" : 41.1008195317
    },
    "center" : {
      "x" : -8.0894566771,
      "y" : 41.0571319714
    }
  }
}

pt-admin-areas-2012-refactor.json

Contains the exported data, one object per line instead of a jsonArray.

pt-admin-areas-transliteration

mongo < pt-admin-areas-transliteration.js

This script will:

• Translitterate Admin Area name into ascii and hyphenise it.
• Create a string containing the full path of an Admin Area, based on tranlsiterated name and ancestor path.

It adds two fields to the documents:

transilterated_name: String
breadcrumb: String