Status Reporting Project

Background and Objective
This was prompted by a Zoom video conference meeting held on Wednesday, May 1, 2024 with 5 members of the Forward Observer group. The original purpose for the meeting was to be a JS8 training and familiarization session, but the discussion soon led to the topic of how to set up a system to collect, collate, analyze, and then distribute status reports from around the country during a grid-down or other situation where “normal” communications are unavailable.

The following is an effort to design a system that would work around the problems we observed in other attempts to build an organized system of collecting information on a large scale. It does not address how that raw data is processed and then distributed.

The key concept in this proposed pilot project is the Tidal Flow model – at any given time, information is flowing in only one direction, with the objective being to reduce data collisions.

This is being proposed as a pilot program. It is an experimental first step. Please use the Comments section below for suggestions, comments, and questions.

Proposed Data Flow for a Nation-Wide Status Report Collection, Analysis, and Distribution System

Rev. 05/05/2024

• Objectives:
  • Flexibility – Must be able to quickly adapt to a rapidly changing situation with new factors to be monitored and old factors no longer important.
  • Simplicity – Local Station requirements must be as simple and as software-independent as possible. The more complex it is, the fewer Local Stations there will be to report on conditions. The records that are reported by Local Stations use a simple spreadsheet model – something that the vast majority of people would be familiar with.
• Collection and Distribution are two very different tasks requiring two very different processes. It cannot be a two-way exchange – they must be done separately.
• There would be three distinct components:
  • JS8 Local stations that collect status and send it into the system
    • The StatReports are very brief, in a delimited format meeting explicit file format.
  • JS8 Regional Collection Stations, set up as a mailbox to receive StatReports
    • East, Central, and West should pretty much cover the country
    • At least one backup station in each region
  • JS8 Central Station with at least one backup station
• Collection and Distribution are done in separate blocks of time, and they do not overlap. At any given time, data is either being collected or distributed – never both.
• In practice:
  • Local stations send their formatted StatReport to their Regional Collection Station on a very rough time schedule to add some load leveling to the collection process.
  • Regional Collection Stations do initial filtering (either deleting or repairing StatReports that do not meet the file specs), then send the daily StatReports in a batch to the Central Station. Probably repeat at least once as a backup.
  • Central Station processes the reports [major factor to be determined] into a single StatReport Summary doc.
  • Central Station sends the StatReport Summary to the Regional Stations.
  • Regional Stations then send the StatReport Summary as a group. Each region uses enough of an offset to avoid interference with other Regional Stations. [Unknown if JS8 is a good choice for this or some other mode. Much will depend on the length of the StatReport Summary.] This is repeated at least once as a backup.
• Record Format / File Specs
  • Each record will include a version identification as the first field.
    
  • The Version ID will include a letter that indicates the total number of fields in a record. This corresponds to the column labels in a standard spreadsheet. The only exception to using the specified number of fields is if all conditions are functioning as normal.
  • Record format is CSV (Comma Separated Value).
  • Record Fields
    • Record format version (includes letter indicating total number of fields)
      
    • Station ID / Call sign submitting report
    • State (2 letter standard)
    • Maidenhead grid (4 character)
    • If all status is Green and no issues to report, then “OK”, in which case this is EOF (End of File) and ready to send. When Central Station processes the records, the “OK” records have the remaining fields appended as Green before further processing.
    • If there are any issues to report, then all fields are used.
    • Each issue is assigned a one-letter ID. These are samples only, since flexibility is a prime objective. Sample Issue-ID:
      • (T)elephone
      • (P)ower
      • (W)ater
      • (H)ospital
      • (I)nternet
      • (R)oads
      • (A)irport
      • (E)mergency Services (EMS, Fire Dept.)
      • (L)aw Enforcement
      • (C)ommunity stability
    • Each Issue-ID is followed by a Status-ID
      • (G)reen – functioning as normal
      • (Y)ellow – significant problems, but semi-functional, not dependable
      • (R)ed – not functioning, but infrastructure remains intact
      • (B)lack – not functioning, infrastructure destroyed,
      • (W)hite – not applicable or unknown
    • Reporting Stations should have the record format set up as a spreadsheet that can be edited, then saved as a CSV file to be copied and pasted into JS8  as Saved Messages.
    • Example Record for Version 2 from station KN4AM in Florida where cell phone, power, and Internet are out (along with associated problems with Emergency Services, Law Enforcement, and Community Stability is unknown:
      • 2N,KN4AM,FL,EL99,TR,PR,WG,HG,IR,RG,AG,EY,LY,CW
  • Fields and Format – The fields are decided upon and distributed by the Central Station, and take the form of a single row CSV file that could be pasted into a spreadsheet as a Header record. This will serve as a guide to the Local Stations to assure record accuracy.
• Notes:
  • This is being described as a daily process, but perhaps a less frequent schedule would suffice – maybe data collection on Monday, Wednesday, and Friday, with StatReport distribution on Tuesday, Thursday, and Saturday.
  • These are Status Reports – not Situation Reports, which, if needed, would go into a completely separate process. The StatReports are more like raw data rather than text describing a situation.
  • JS8 is used for the collection process. The digital mode for the distribution process is an unknown, but JS8 is still a possible solution. It depends on the length of the compiled StatReport Summaries.
  • SOI would call for transmitters to be turned off unless they are sending their StatReport.
  • The system should take full advantage of automation features of JS8Call.
  • There are no acknowledgements, SNRs, Heartbeats, etc. Information sharing follows a “Tidal Flow” model – during certain hours/days, all information is flowing into the Collection Stations, then during a different time slot, all information is flowing out to the Collection stations for distribution in their respective regions. This is not a typical two-way QSO – it is purely moving large amounts of data in one direction or the other.