-
Notifications
You must be signed in to change notification settings - Fork 8
Avro Packet Maker
Overview
This processor works with a table (dataframe) of candidates to generate an alert packet for each candidate. The alerts are formatted as Apache Avro, which is a data serialization system that relies on schemas to validate and enforce data types and contents.
Alert Schema
Packets are made based on a nested schema format, similar to that used by ZTF.
Schemas are in processors/alert_packets/alert_schema/
and are given as .avsc files. alert.avsc
is the top-level schema. candidate.avsc
and prv_candidate.avsc
are packed into alert.avsc.
Requires python 3 and avro.schema
for combining nested schemas. This module specifically works only with avro version 1.10.1 (which you can get with pip install avro==1.10.1
). Also uses fastavro
library, which is available with pip install fastavro
.
Saving Packets Locally
If Avro packets are being saved locally, ensure that the output data directory has been set correctly. Use export OUTPUT_DATA_DIR=/path/to/directory
to set.