Skip to content
Ben Leighton edited this page Sep 23, 2019 · 11 revisions

This page documents discussion and design of a Loc-I Integration API

  1. Name

Loc-I Integration API

...

1. Goal/scope

Complement existing Loc-I  (see v2 component architecture) with the addition of a Loc-I Integration API (preferably OpenAPIv3) for these purposes.

Primary purposes

  1. Provide a uniform interface/standardise access to Loc-I cache resources for common operations (without needing to know SPARQL)
  2. Separate front-end/client-application-specific logic from query logic of common operations (Integration API interfacing with Application-specific APIs)
  3. Maintain a separation between API and backend implementations (changes in data models, triplestore/database implementations, load balancing, etc)
  4. Enable rapid development of new applications from reuse of query logic of common operations

Secondary purposes

  1. Provide a platform to facilitate open-source development of additional APIs
  2. Cache common operations for performance
  3. Allow API deployment in separate cache configs
  4. ... (probably others)

Future design goals

  1. Federated queries, queries across multiple caches

\

  1. User stories


User story 1 lookup location


As a/an... Enterprise data warehouse manager

I want to... a) Find spatial identifier(s) for a location (geospatial feature) in my database denoted by a (i) name (ii) text description (iii) lat/long, DGGS, or GPS location

             b) Add location information for a new entry for an observation or data point based on some location description (text)

so that... I can ensure consistency across my data warehouse and provide opportunities to integrate with other data

\


User story 2 correlate with other indexed locations


As a/an... Spatial data analyst

I want to... a) For a location (geospatial feature) in dataset A find which location(s) (geospatial feature(s)) from dataset B are contained within it 

             b\) map observations/data from one geography (Geofabric) to reporting
             units in another geography (ASGS) (reapportionment)
             
             c\) For a location (geospatial feature), find related locations
             (geospatial feature) within the dataset by some relationship (within,
             contains, overlap)

so that... I can understand influences from other domains on the location (geospatial feature) I'm interested in and retrieve related locations

\

  1. Issues/concerns 

1. What is deemed as common operations in the Integration API bucket?

2. Communicating that the Integration API is a complement to the SPARQL API - full power of Loc-I and the Linked Data features

\

  1. API design and implementation

\

What do we design and implement as the Integration API?

Core Integration API endpoints

+------------+------------+------------+------------+------------+------------+------------+ | Endpoints | User story | Priority | HTTP | Parameters | Expected | Notes | | | | | | | response | | | | | (1 = | | | | | | | | highest | | | | | | | | priority, | | | | | | | | 3=lowest) | | | | | +============+============+============+============+============+============+============+ | https:// | 2 | 1 | GET | uri={lociU | List of | Prefer | | api.loci.c | | | | ri} | URIs | 'location' | | at/v1/obje | | | | | denoting | rather | | ct/contain | | | | Areas='Tru | things | than | | s | | | | e' | that | 'object' | | | | | | | contains | as it is | | https://ap | | | | includePer | lociUri | consistent | | i.loci.cat | | | | centage='T | locationTh | with the | | /v1/locati | | | | rue' | ing | definition | | on/contain | | | | | | e.g. | | s | | | | FullCont | | prov:Locat | | | | | | ainment='F | | ion | | | | | | alse' | | | | | | | | | | Prefer | | | | | | page={page | | exclusive | | | | | | num} | | contains. | | | | | | | | | | | | | | offset={of | | | | | | | | fsetnum} | | | | | | | | | | | | | | | | limit={num | | | | | | | | results} | | | +------------+------------+------------+------------+------------+------------+------------+ | <span | 2 | 1 | GET | uri={lociU | List of | For | | class="nol | | | | ri} | URIs | partial | | ink">https | | | | | denoting | and full | | ://api.loc | | | | Areas='Tru | things | contains/w | | i.cat</spa | | | | e' | that | ithins  | | n>/v1/obje | | | | | overlaps | | | ct/overlap | | | | includePer | lociUri | | | s | | | | centage='T | locationTh | | | | | | | rue' | ing | | | <span | | | | | | | | class="nol | | | | FullContai | | | | ink">https | | | | nment='Fal | | | | ://api.loc | | | | se' | | | | i.cat/v1/l | | | | | | | | ocation/ov | | | | page={page | | | | erlaps</sp | | | | num} | | | | an> | | | | | | | | | | | | offset={of | | | | | | | | fsetnum} | | | | | | | | | | | | | | | | limit={num | | | | | | | | results} | | | +------------+------------+------------+------------+------------+------------+------------+ | ~~<span | 2 | 1 | GET | uri={lociU | List of | Prefer | | class="nol | | | | ri} | URIs | exclusive | | ink">https | | | | | denoting | within. | | ://api.loc | | | | Areas='Tru | things | | | i.cat</spa | | | | e' | that are | | | n>/v1/obje | | | | | within | | | cts/within | | | | includePer | lociUri | | | ~~ | | | | centage='T | locationTh | | | | | | | rue' | ing | | | <span | | | | | | | | class="nol | | | | page={page | | | | ink"><span | | | | num} | | | | class="nol | | | | | | | | ink">https | | | | offset={of | | | | ://api.loc | | | | fsetnum} | | | | i.cat</spa | | | | | | | | n>/ | | | | limit={num | | | | v1/locatio | | | | results} | | | | n/within | | | | | | | +------------+------------+------------+------------+------------+------------+------------+ | ~~<span | 2 | 1 | GET | <span | Descriptio | Describe | | class="nol | | | | class="nol | n | query to | | ink"><span | | | | ink">uri={ | of the | the triple | | class="nol | | | | lociUri}</ | object | store | | ink">https | | | | span> | resource | | | ://api.loc | | | | | denoted by | | | i.cat</spa | | | | | the | | | n>/v1/obje | | | | | lociUri. | | | ct~ | | | | | This could | | | ~ | | | | | be an | | | | | | | | instance | | | <span | | | | | of | | | class="nol | | | | | (geo:Featu | | | ink"><span | | | | | re | | | class="nol | | | | | or | | | ink">https | | | | | prov:Locat | | | ://api.loc | | | | | ion) | | | i.cat</spa | | | | | or | | | n>/v1/reso | | | | | loci:Links | | | urce</span | | | | | et | | | > | | | | | | | +------------+------------+------------+------------+------------+------------+------------+ | <span | \ | ? | GET | page={page | List of | Might be | | class="nol | | | | num} | URIs | an easy | | ink"><span | | | | | denoting | endpoint | | class="nol | | | | offset={of | things | to | | ink">https | | | | fsetnum} | that | implement | | ://api.loc | | | | | are of | | | i.cat</spa | | | | limit={num | type | | | n>/v1/loca | | | | results} | geo:Featur | | | tions</spa | | | | | e | | | n> | | | | | or | | | | | | | | prov:Locat | | | | | | | | ion | | +------------+------------+------------+------------+------------+------------+------------+ | <span | \ | ? | GET | page={page | List of | Might be | | class="nol | | | | num} | URIs | an easy | | ink"><span | | | | | denoting | endpoint | | class="nol | | | | offset={of | things | to | | ink">https | | | | fsetnum} | that are | implement | | ://api.loc | | | | | of type | | | i.cat</spa | | | | limit={num | loci:Links | | | n>/v1/link | | | | results} | et | | | sets</span | | | | | | | | > | | | | | | | +------------+------------+------------+------------+------------+------------+------------+ | <span | \ | ? | GET | page={page | List of | Might be | | class="nol | | | | num} | URIs | an easy | | ink"><span | | | | | denoting | endpoint | | class="nol | | | | offset={of | things | to | | ink">https | | | | fsetnum} | that are | implement | | ://api.loc | | | | | of type | | | i.cat</spa | | | | limit={num | loci:Datas | | | n>/v1/data | | | | results} | et | | | sets</span | | | | | | | | > | | | | | | | +------------+------------+------------+------------+------------+------------+------------+ | <span | 1 | 2 | GET | name={quer | Query by | Option | | class="nol | | | | y | text | 1. Could | | ink"><span | | | | text} | (multiple | implement | | class="nol | | | | | fields) | this as a | | ink">https | | | | dataset={d | for a | sparql | | ://api.loc | | | | ataset | location | regex | | i.cat</spa | | | | name, e.g. | (geospatia | query if | | n>/v1/obje | | | | asgs2016} | l | harmonised | | ct/find-by | | | | | feature) | to | | -name</spa | | | | | | prefLabels | | n> | | | | | | for all | | | | | | | | names. | | <span | | | | | | Requires | | class="nol | | | | | | work to | | ink"><span | | | | | | harmonise | | class="nol | | | | | | how labels | | ink">https | | | | | | are | | ://api.loc | | | | | | represente | | i.cat</spa | | | | | | d. | | n>/v1/loca | | | | | | | | tion/find- | | | | | | Option | | by-name</s | | | | | | 2. | | pan> | | | | | | Implement | | | | | | | | this as a | | | | | | | | sparql | | | | | | | | regex | | | | | | | | query on | | | | | | | | the | | | | | | | | multiple | | | | | | | | text label | | | | | | | | fields. | | | | | | | | Hackish | | | | | | | | option to | | | | | | | | option-1.  | | | | | | | | | | | | | | | | Option | | | | | | | | 3. Else, | | | | | | | | might need | | | | | | | | to think | | | | | | | | about | | | | | | | | indexing | | | | | | | | this | | | | | | | | elsewhere | | | | | | | | e.g. | | | | | | | | elasticsea | | | | | | | | rch. | +------------+------------+------------+------------+------------+------------+------------+ | <span | 1 | 3 | GET  | lat={lat} | Query by | Handy but | | class="nol | | | | | point for | might | | ink"><span | | | | long={long | a location | require a | | class="nol | | | | } | (geospatia | spatial db | | ink">https | | | | | l | engine to | | ://api.loc | | | | filter={da | feature) | return | | i.cat</spa | | | | taset | | answers | | n>/v1/obje | | | | name eg. | | | | ct/find-by | | | | asgs2016} | | | | -point</sp | | | | (optional) | | | | an> | | | | | | | | | | | | | | | | <span | | | | | | | | class="nol | | | | | | | | ink"><span | | | | | | | | class="nol | | | | | | | | ink">https | | | | | | | | ://api.loc | | | | | | | | i.cat</spa | | | | | | | | n>/v1/loca | | | | | | | | tion/find- | | | | | | | | by-point</ | | | | | | | | span> | | | | | | | +------------+------------+------------+------------+------------+------------+------------+ | <span | 1 | 4 | GET  | polygon={p | Query by | Handy but | | class="nol | | | | } | polygon | might | | ink"><span | | | | | for a | require a | | class="nol | | | | | location | spatial db | | ink">https | | | | | (geospatia | engine to | | ://api.loc | | | | | l | return | | i.cat</spa | | | | | feature) | answers | | n>/v1/loca | | | | | | | | tion/find- | | | | | | | | by-polygon | | | | | | | | | | | | | | | +------------+------------+------------+------------+------------+------------+------------+

\

Unclear if in Loc-I Integration API {#unclear-if-in-loc-i-integration-api .auto-cursor-target}

+--------------------+--------------------+--------------------+--------------------+ | Endpoints | HTTP | Parameters | Expected response | +====================+====================+====================+====================+ | <span | GET | from={lociUri} | Value with the | | class="nolink"><sp | | | answer of the | | an | | to={lociUri} | reapportionment | | class="nolink">htt | | | function | | ps://api.loci.cat< | | value={some value} | | | /span>/v1/o | | | | | bject/reapportion | | | | +--------------------+--------------------+--------------------+--------------------+ | <span | \ | \ | \ | | class="nolink"><sp | | | | | an | | | | | class="nolink">htt | | | | | ps://api.loci.cat< | | | | | /span>/v1/sparql</ | | | | | span> | | | | +--------------------+--------------------+--------------------+--------------------+

How do we implement the Integration API?

Since GA and CSIRO teams are familiar with Python, first cut choose a Python implementation. Something that can help automate generation of a swagger doc/api for things implemented in the Integration API.

\

Action items

2 incomplete Leighton, Ben (L&W, Clayton) to coordinate the team (Sommer, Ashley (L&W, Dutton Park) Cox, Simon (L&W, Clayton) Seaton, Shane (L&W, Black Mountain) ) to work on v1 development and testing of loci integration API and refactoring of appropriate bits of excelerator/iderdown.  

\

References

  1. **Loci journey maps **https://drive.google.com/drive/folders/1Hg19o5X92d173AkFg3vVhe5dp5Xd_Nuv

  2. **AbnLookup **https://abr.business.gov.au/Tools/AbnLookup

    1. https://abr.business.gov.au/Search/ResultsActive?SearchText=farm
    2. https://abr.business.gov.au/Search/ResultsActive?SearchText=water
  3. **Anzsic code lookup **https://www.ato.gov.au/Calculators-and-tools/Business-industry-code-tool/AnzsicCoder.aspx

    1. e.g. https://www.ato.gov.au/Calculators-and-tools/Business-industry-code-tool/AnzsicCoder.aspx?s=fish

\

\

\

Clone this wiki locally