Report of Radio Skill Drill 2342

C. Matthew Curtin, KD8TTE

kd8tte@pm.me

25 October 2023

Introduction

Can we get reliable and specific ground-truth information for planning response to emergencies?

That's the question we set out to answer when we launched a no-notice drill on the evening of 16 October 2023. Over the course of the week that followed, we issued new requests for information at irregular intervals and collected responses.

The exercise is part of the ongoing training program delivered in connection with the BLACK SWAN COMEX and its sponsoring organization, the Ohio Military Reserve, part of the State Defense Force Command of The Adjutant General's Department, State of Ohio.

Trained operators were able to provide ground-truth observations regarding infrastructure, specifically-requested information, and local weather information in standardized data form that could be automatically compiled and used by the receiver as required.

Methodology

Operation of the exercise had five components, each of which was part of earlier training. We show here our methods for Activation, Tactical Messaging, Message Transmission, Message Processing, and Presentation.

Activation

One of the most difficult issues in our experience in working with amateur radio service volunteers is activation. The BLACK SWAN COMEX in 2020 suffered serious failures in the partner NGO's affiliates to follow instructions and to activate resources that were themselves readied and standing by to engage.

Subsequent NGO inaction to correct the problem was unsatisfactory and to ensure that volunteer resources could be engaged, we changed our engagement model to require each volunteer individually to be registered with the agency and to be managed directly by the agency.

We included training on how to activate a station, prepare for AUXCOMM operations, and to mobilize when requested in our weekly training at various points in years 2022 and 2023.

• Training 2228 Station Activation. https://groups.io/g/QTC/message/768

• Training 2305 Prepare for AUXCOMM Operations. https://groups.io/g/QTC/message/873

• Training 2311 Mobilization and Arrival. https://groups.io/g/QTC/message/904

Another part of our regular training is operation of the QSP Letter, a newsletter sourced and distributed entirely by radio. That newsletter is distributed initially via the BLACK SWAN Net via HF. Subsequent distribution takes place through relay stations, including through the Winlink Global Radio Email System. Weekly training (numbered tasks or events) are released in an article in the QSP Letter. The following morning, the training article from the QSP Letter is published on the QTC Internet mailing list for tracking and discussing weekly training.

Participants in the exercise were given no prior notice. The first announcement of the drill took place in QSP volume 3 number 42, with subsequent announcement on the QTC mailing list as Task 2342.

Tactical Messaging

By tactical messaging, we mean communication of specific information to aid a larger operation. That information is typically captured in a specific format to support brevity and precision in the message content, and flexibility in transmission method, allowing for information to flow along needed lines, adapting to the conditions of the operation.

Regular practice of tactical messaging takes place in our training program by highlighting tools, techniques, and procedures for meeting various messaging needs.

• The QTC Mailing List is specifically organized around the concept of Radio Messaging.

• Training 2244 Using FLMSG Custom Form. https://groups.io/g/QTC/message/843

• Training 2231 Basic Message Format. https://groups.io/g/QTC/message/780

• Training 2329 Install FLMSG Custom Form. https://groups.io/g/QTC/message/987

• Training 2320 Send Field Situation Report (FIELDSITREP). https://groups.io/g/QTC/message/993 

• Training 2335 Send Weather Observation [WXOBS]. https://groups.io/g/QTC/message/1008

For the purpose of this exercise, Information A through E directed responses to come in the form of FIELDSITREP, and Information F requested WXOBS.

Message Transmission

Our training program likewise includes regular practice of message transmission. The QSP Letter is transmitted weekly via radio, as are other messages from various sources including messages for all stations in the net, groups of stations, and specific station-to-station messaging.

Transmission practice includes both the Winlink Global Radio Email system and at least thrice-weekly operation of the BLACK SWAN Net. Statistics for the net include the number of message transmissions that take place. (For January-September 2023, over 3500 messages of various precedence were acknowledged.)

• Training 2333: Report Into the Net With Winlink. https://groups.io/g/QTC/message/1003

• Training 2327: Send With FLAMP. https://groups.io/g/QTC/message/981

• Training 2326: RELAY MESSAGE WITH FLAMP. https://groups.io/g/QTC/message/974

In this exercise, transmission instructions variously included using Winlink, BLACK SWAN Net (which uses procedures 2327 and 2326), and even allowed for attachment of the formatted message to be sent by Internet email through Winlink, where the exercise controller would receive the message via Winlink over HF.

Message Processing

All messages, whatever transmission method used, eventually routed by radio to Bexley Station for processing. The requesting agency publicly presents the FLMSG Custom Forms for FIELDSITREP and WXOBS messages, and also has back-end processing software to make use of received messages.

Rather than opening the received messages in the same form for on-screen viewing, the agency uses the back-end software to convert the formatted data to a tab-separated value (TSV) file. Each message results in a single TSV-formatted row, with columns consistent for each message type. Thus, each FIELDSITREP can be processed, with results concatenated together to a single TSV file of FIELDSITREPs. The same can be done for WXOBS messages to create a single TSV file of WXOBS reports.

Presentation

As Bexley Station received reports, the TSV files for the reports were updated, allowing for a view of the data "as of now." Raw data were available either as plain text, or could be easily imported to a spreadsheet.

For reporting purposes, responses were stored in a spreadsheet with separate tabs for each information request A-F. Columns were determined by report format (FIELDSITREP or WXOBS), and each row stored a different report. Each tab therefore showed all responses to a particular information request.

Presentation required two manual steps. The first was to ensure that each report had the position reported in the form of latitude and longitude. FIELDSITREP and WXOBS messages are intended for field use, where operators might not have ready access to reliable coordinates. Operators who know grid coordinates as MGRS or Maidenhead rather than latitude and longitude may report accordingly, on the theory that the processing station at a command post will have more ready access to maps and tools needed for conversion if needed.

The second manual step was to make corrections if needed, e.g., converting longitude reported in degrees without correctly specifying west.

Results

The no-notice drill resulted in reports from eight stations in addition to Bexley Station KD8TTE, which both controlled the exercise and made reports. The complete list of stations reporting is AD8CM, K8BCI, KC9TYA, KD8TTE, W8JTW, W8KVK, WD8IIJ, WD8SAB, and WD8SDH.

Not all stations reported for all objectives. Participants engaged on a volunteer basis, with no prior notice, on a best-effort basis, while juggling normal demands of daily schedules.

We received a total of 29 reports, with a maximum of nine reports, and minimum of three reports for each information type. This gives us an average of 4.83 responses per request. Information D (radio services available) got a response from all participants. The first and last requests (A and F) had the smallest period of time the request was open and the lowest response rate, with three each.

Some of the reports did not specify the information designator (A-F) but could be disambiguated by the additional information reported. Some reports did not include the date-time group as part of the message. All operators with the net's DRYAD table correctly authenticated their messages.

An interactive map of the reports follows. Click on any point to show the contents of the report. Each information request A-F is color-coded, with the icon intended to give the viewer an idea of what information was reported in "additional" information. Weather observations are yellow, with the icon indicating the level of cloud coverage reported. 

Discussion

We return to our original question: Can we get reliable and specific ground-truth information for planning response to emergencies?

Clearly, the answer is "yes." We note that volunteer radio operators were able to provide specifically-requested information in specifically-requested format that allowed for easy aggregation and processing.

Comparison of these results to results in other exercises larger and smaller, sponsored by this agency and others, shows several elements are critical.

1. Doctrine to support agency needs.

2. Standardization at interface layers.

3. Well-specified procedures.

4. Practice.

5. Improvement plans, including updates to doctrine, standards and procedures.

We briefly discuss each in turn.

Doctrine to Support Agency Needs

Doctrine is a body of beliefs and instructions that work together to some end. In our case, we are looking specifically at the matter of how to allow amateur radio operators and more generally volunteers providing communications services to support security and resilience of their communities, and collectively the nation.

Whatever an agency's mission, there must be a common foundation for structuring and conducting operations. Too often, both the agencies conducting operations and people providing communications services (whether internal or in the form of some kind of auxiliary) simply assume that possession of a radio license has provided some qualification for communicating. This is simply false: the holder of an Amateur Extra license doesn't necessarily know how to transmit an ICS Form 204 on behalf of an agency in a way that the recipient represented by another Amateur Extra licensee will be able to make sense of it.

Real-world disaster response scenarios and exercises show time and again that unless there is good doctrine, people will find themselves at cross-purposes and while all attempting to advance the mission, waste tremendous time and effort before communication can be established.

Standardization at Interface Layers

In the wake of the "9/11" attacks on the United States, a commonly-cited failure was the inability for emergency responders from different services to communicate. The result has been a push to "standardize" in an effort to allow for "interoperability." Such efforts are typically addressed by vendors who want to sell a "solution" that addresses enumerated requirements and addresses interoperability concerns by requiring that all agencies buy from the vendor's product line. One vendor's line may or may not work with another. The result not only makes the buyer beholden to the seller but creates a single point of failure, as we see when centrally-managed P25 radio systems get overwhelmed when many responders are in an area. (Ohio's Multi Agency Radio Communications Service - MARCS - is a good example. When agencies from around the state send representatives to a fallen officer's funeral, MARCS circuits often experience overload.)

Requirements to address this have led to yet another round of incorrectly-specified standards in FirstNet. By promising high availability and priority access, the single system has been seen by many agencies around the country as a "solution" to the problem of communication. Many expressed surprise when a single VBEID in Nashville on Christmas Day 2020 took out communication circuits for five states, leaving some FirstNet-dependent law enforcement agencies without service for more than twenty-four hours.

As we have seen through the development of the Internet, success allows for each user to choose their own applications, operating systems, and computer platforms. Any system can make use of the entire system by standardizing at the network layer around the Internet Protocol, and can support the rest of the suite including packet types like the Transport Control Protocol, services specified openly like the Simple Mail Transfer Protocol, and message formats like the Internet Message Format.

Communications built around standardized and openly-specified message formats allow for the origination, relay, and delivery of information for any purpose, with each transmission using the means of propagation, waveform, and techniques that best suit the circuit in use.

The BLACK SWAN Net makes use of the openly-specified MFSK waveform, and open-source software such as FLDIGI, FLMSG, and FLAMP. These software packages are available for practically any platform, and other applications are free to implement the waveform or amateur multicast protocol to allow for message relay. The net furthermore makes use of gateway services to the Winlink Global Radio Email system so that stations needing to communicate asynchronously, over distances not supported by the HF circuits in use, or with another waveform may engage with the net by relaying through the Winlink system.

This allows for resilience that cannot be found with single-vendor "solutions" or other options that require all operators use the same software.

Well-Specified Procedures

Procedures that establish the objective, the conditions under which the procedure may be started, and the tasks to be performed to accomplish the objective are critical. Many operators erroneously claim to "know how" to do something like send a message. Yet their use of voice procedure shows inefficient use of the circuit, ambiguous instructions, or worse yet, garbling of message content. These are all symptoms of a failure to use good procedure.

Operators cannot use good procedure if they don't have a good reference for it, and which set of procedures to use. The BLACK SWAN Net has adopted standards like ACP 124, 125, 126, and 131 to standardize on message format, procedure words, and procedure signals. Likewise we have adopted ACP 121 for general instructions that include details like quality of service expectations as determined by message precedence, which conveniently is the same standard as used by US Government radio services and the Winlink Global Radio Email system.

Additional procedures are needed for practical purposes, to get the daily work of the net done. That's where our weekly training comes into play. We define tasks, when to perform them, and how to perform them. Our planners are able to build lists of tasks needed for performing specific operations, and our volunteers are able to train to those standards.

Practice

Skill is developed through practice. The best procedures are not useful if not practiced. Not only is repetition needed to learn a skill in the first place but to maintain it. 

The BLACK SWAN Net operates at least thrice weekly. This provides operators with varying schedules and availability many opportunities to practice the procedures included in weekly training and regularly-conducted exercises.

Improvement Plans

We have seen that amateur radio volunteers often expect congratulations for showing up. While appearance is commendable and certainly necessary, training and improvement requires critical assessment of performance. Procedures may need correction, practices may need correction, and people may need instruction. Unless operators participate with an open mind, offering and receiving constructive criticism, improvement is impossible.

Creating a list of "things that didn't go well" is insufficient, however. Criticism must be followed by action. First, assessment (is the criticism correct?), and then a course of action for any accepted criticism. Do procedures need updating? To training materials or methods need to be changed? Do people need to spend time in training?

Notice that BLACK SWAN COMEX annual After-Action Report/Improvement Plans (AAR/IP) include actions for improvement. Experience shows that we almost never specify something correctly the first time but through practice we make necessary modifications that feed back into our doctrine, which in turn are reflectedin operation.

Conclusion

The Amateur Radio Service and its volunteers can be, but are not necessarily, valuable contributors to community security and resilience. Value does not come from badge-collecting, advanced licensing, expensive gear, or titles. Value comes from regular practice in a context with good doctrine, standards, and procedures.

We encourage all amateur radio licensees with an interest in public service to engage: whatever your experience, license, or gear is, it is only a place to start. Training programs such as ours at the BLACK SWAN COMEX are the critical difference between proven viability and effort that may not be put to most productive use.

Appendix: Instructions

Participants were directed to a specified URL to pick up exercise information. The URL was unpublished, so the only way to get it was to get the start of exercise information. At various points throughout the week, the instructions changed. We provide all instructions given, along with information designator (A through F), and the date-time group that the information was published.

Information: A - Issued 170809Q Oct 2023

1. Complete FIELDSITREP indicating conditions at your station.

2. In Line 12 of the FIELDSITREP, indicate the frequency and status of the amateur radio repeater you monitor most.

3. Save the FIELDSITREP and send the resulting k2s file by Winlink to BLACKSWAN or by Internet email to BLACKSWAN@Winlink.org.

Information: B - Issued 171900Q Oct

1. Complete FIELDSITREP indicating conditions at your station.

2. In Line 12 of the FIELDSITREP, indicate information B and a well-known landmark in your area.

3. Save the FIELDSITREP and send the resulting k2s file by Winlink to BLACKSWAN or by Internet email to BLACKSWAN@Winlink.org.

Information: C - Issued 182025Q Oct

1. Complete FIELDSITREP indicating conditions at your station.

2. In Line 12 of the FIELDSITREP, indicate information C, the airport closest to you, and how long it takes to get there by car.

3. Save the FIELDSITREP and send the resulting k2s file by Winlink to BLACKSWAN or by Internet email to BLACKSWAN@Winlink.org.

Information: D - Issued 191700Q Oct

1. Complete FIELDSITREP indicating conditions at your station.

2. In Line 12 of the FIELDSITREP, indicate information D and the radio services available at your station, e.g., FRS, MURS, CB, GMRS, LMRS, Maritime Mobile, SHARES, etc.

3. Save the FIELDSITREP and send the resulting k2s file by Winlink to BLACKSWAN or by Internet email to BLACKSWAN@Winlink.org.

Information: E - Issued 211711Q Oct

1. Complete FIELDSITREP indicating conditions at your station.

2. In Line 12 of the FIELDSITREP, indicate information E and quality of Internet service as indicated by https://www.speedtest.net/. Provide Download speed, Upload speed, and ping times, e.g., Download 50 Mbps, Upload 20 Mbps, Ping ms 20, 100, 50.

3. Save the FIELDSITREP and send the resulting k2s file by Winlink to BLACKSWAN or by Internet email to BLACKSWAN@Winlink.org.

Information: F - Issued 221222Q Oct

1. Complete WXOBS indicating conditions at your station.

2. In Line 10 of the WXOBS, indicate information F and whether you expect to be on the air for the WILDCARD 23 exercise on 4 Nov.

3. Save the WXOBS and send the resulting k2s file to NNO5HD by BLACK SWAN Net, by Winlink to BLACKSWAN, or by Internet email to BLACKSWAN@Winlink.org.

Appendix: Reports

This appendix shows the spreadsheet where processed reports were aggreated. Tabs are for Raw (direct results from processing software), Cooked (data corrected or augmented with coordinates where position was reported by rid), and then by information response A-F.

(NB reports from KD8TTE were provided at the end of the exercise and may appear directly in a specific tab without being in Raw. Those reports were generated at the station locally with the FLMSG software and processed, as with all other stations, but results were in some cases pasted directly into the information response tab.)