Bulk Pack

Bulk Pack Processing v2

Field data schem

Ingestion Process by Entity Type

wallet_registration

Wallet registrations need to be stored in the field data schema in the wallet_registrations table, but they also need to be used to upsert (create or update) grower account records in the treetracker API. A wallet registration also needs the grower account id field populated.

Suggested workflow:

1. Bulk pack transformer calls PUT on grower_account in treetracker API for grower account with identifier that matches wallet

   1. If account exists, update

      1. Only update the name, phone, and email for now

   2. Else, insert

      1. Insert identifier, name, phone, email, and photo

2. POST to /planter path on bulk-pack-transformer-1 to populate legacy schema

3. Populate grower account id in wallet registration data, and POST into field data api

   1. If wallet registration id is already present, field data api will do nothing (return 200)

4. Return 200

device_configuration

device configuration records are simply inserted in v2, but in v1 they were upserted.

Suggested Workflow

1. POST device configuration payload to bulk-pack-transformer-1

   1. bulk-pack-transformer-1 returns 200 if already present

2. POST device configuration payload to field data API

   1. field data API returns 200 if already present

3. Return 200

session

session does not exist in the v1 bulk-pack-transformer.

Suggested Workflow

1. POST to session path on field data API

   1. field data API returns 200 if already present

2. Return 200

capture

1. POST capture data to bulk-pack-transformer-1

   1. If data exists this API returns 200, else a new record is inserted

   2. This API also propagates the record to field data API, where a queue message is emitted

2. POST capture data to field data API

   1. Since step 1 already propagated to this API, a 200 should be returned based on uuid

   2. Since 200 is returned, queue message should not re-emitted

Orchestration Diagram

Bulk Pack Format v2 : https://app.gitbook.com/o/-MXNadx4i6aOZ12XcStA/s/-MXtAguKaWMpiXXl0UDb/upload-pack-format

Improved orchestration architecture

Deprecated proposal for orchestration

Nullable Columns for Field Data Tables

Session

• track_url

• organization

Wallet Registration

• phone or email is nullable, but not both

Raw Capture

• session_id is nullable for v1 endpoints only

• reference_id

• note

• extra_attributes

• rejection_reason

Device Configuration

• No nullable fields

Workflow Notes

Insert 'tree' as 'capture' will fail until corresponding device and registration records have been inserted, because there's no session Once device and registration records are inserted, then it is possible to insert a session id using SESSION UUID = device_identifier + planter_identifier In v2 bulk pack endpoint /v1/tree is responsible for ensuring that the session record exists for a v1 tree. This session can then be used as the session for inserting the v1 'tree' record as a v2 'capture' record

1. process sends tree record to /v1/tree in v2

2. v2 calculates SESSION UUID = device_identifier + planter_identifier using uuid-string

3. v2 checks for a session that matches this

4. If no session exists, it looks up device_configuration_id and wallet_registration_id using device_identifier and planter_identifier, and if they exist it creates a session record. If they don't exist yet, it fails until next time.

5. Assuming a session exists or was inserted in (4), we can now insert the capture record using his session uuid.

Release

Old versions:

• greenstand/bulk-pack-processor:1.2.7

• greenstand/bulk-pack-transformer:1.6.2