One was from Chick-fil-A<\/a>, a popular chicken sandwich chain, which runs a Kubernetes-based edge computing platform<\/a> for its 3,000+ outlets across the U.S. The other expert was from the U.S. Air Force, which has among its other duties delivering connectivity and associated core services to remote parts of the world.<\/p>\n \u201cThe edge all depends on who you need to serve, how many people you need to serve, what you\u2019re serving,\u201d explained Michael Henry<\/a>, chief of information technology for the Secretariat of the Air Force, studies and analysis.<\/p>\n Edge is all about bringing \u201ccompute to where the action is,\u201d further explained Brian Chambers<\/a>, chief architect for Chick-fil-A.<\/p>\n The two speakers are also members of Edge Monsters<\/a>, a group of edge architects who meet on the regular to discuss best practices around this unique form of computing.<\/p>\n \u00a0<\/p>\n Chick-fil-A\u2019s tech stack, built largely on open source.<\/p>\n The Edge Is Defined by Constraints<\/p>\n Henry said, \u201cThe edge is something you need to live in, in order to deploy it.\u201d Although there are commercial and open source solutions to help build a stack, each edge system is something unique, defined by both its restraints and requirements.<\/p>\n Chick-fil-A was driven to build an edge platform to support the heavy business some of its stores were seeing, which was leading to unnecessary capacity and logistics challenges.<\/p>\n At least the restaurants stayed put.<\/p>\n The Air Force\u2019s edge systems had to be deployed all over the world on a moment\u2019s notice.<\/p>\n \u201cWe need to be able to pack it up, start it up and get our network connectivity and our core services up and running at a moment\u2019s notice,\u201d Henry said.<\/p>\n The two organizations\u2019 operations are different in other ways as well. The Air Force has a lot of legacy equipment to support, whereas Chick-fil-A makes it a point to standardize the hardware<\/a> across all its locations.<\/p>\n Nonetheless, there are commonalities across most edge systems<\/a>.<\/p>\n The chief characteristic of an edge system is that it has pretty serious constraints of one type or another. Unlike the cloud, with its unlimited scalability (in theory), the edge is practically defined by constraints: physical constraints, network constraints and power constraints.<\/p>\n Edge locations may not have on-site tech support. Space is usually at a premium at most locations. So is power. And then there are financial constraints. One solution may work well for 1 or 10 locations, but does it get unduly expensive for 1,000 or 100,000 locations?<\/p>\n Different systems also have differing requirements.<\/p>\n The Air Force, for instance, requires multiple levels of connectivity for its remote outposts. It also has some \u201cdeep security concerns,\u201d in terms of preventing hostile actors from compromising the system, Henry said.<\/p>\n The requirement is to \u201cdeploy the edge such that we know what is running on this box. And there is absolutely nothing that\u2019s unintentional being able to push patches to this thing,\u201d he said, explaining the server itself is probably an unmanned relay station and \u201czip-tied\u201d to a tree or powerline somewhere.<\/p>\n Cluster configuration is also a huge deal for the military service. The armed service needs to push an update to an edge node and have that go live across the fleet.<\/p>\n All of which comes down to one thing, according to the duo: You own the entire stack.<\/p>\n 6 Principles for Building Edge Systems<\/p>\n In their keynote talk, the engineering duo shared what they called \u201csix principles for crushing it at the edge.\u201d<\/p>\n 1. Do the Differentiated Heavy Lifting<\/p>\n When building an edge system, \u201cNo one is going to do the work for you,\u201d Henry said. There may be some \u201cturnkey solutions\u201d that can help along the way, but in the end, the system you will design will be unique to your operating characteristics.<\/p>\n What will the infrastructure be? How will it \u201cfan out\u201d across all the end nodes? What are your end points? How are you booting? What is your content delivery network?<\/p>\n Also, keep in mind, you can\u2019t bring every service from the cloud over to the edge: They won\u2019t fit.<\/p>\n \u00a0<\/p>\n 2. Build a \u2018North Star\u2019 Edge Controller<\/p>\n A considerable chunk of your time will be dedicated to sorting out the edge controller.<\/p>\n \u201cThere are just so many edge controllers out there that you can play with. But they all have their quirks. They all have their problems. Maybe it\u2019s this Raspberry Pi 5,<\/a> but you got the wrong board, or, like, the board itself doesn\u2019t really work for you exactly right. Am I going to power it correctly?\u201d Henry said.<\/p>\n You have to figure out how to \u201ckick-start\u201d the controller if it goes out, especially if the engineering team is 3,000 miles away.\u00a0 A microSD card can be used to reimage a system, which saves the cost and time of sending an engineer to the remote location; a local operator can do the reboot.<\/p>\n \u201cBecause if you have thousands of these things, getting a pre-burn-in contract so that you delay it gives you a supply chain problem,\u201d Henry said.<\/p>\n 3. Use Declarative Infrastructure<\/p>\n \u201cIf it\u2019s not declarative, it doesn\u2019t exist,\u201d Chambers said. One rule of Chick-fil-A is that all deployments must follow the practice of Infrastructure as Code (IaC)<\/a>, where all software configuration is predefined so it can be rolled out automatically.<\/p>\n Chambers knew from the beginning that, given the number of Chick-fil-A outlets, IaC would be necessary. Once developers started to make changes to their applications, deploying that software had to be done in a uniform and documented manner, lest each system \u201cdrift\u201d into its own unique configuration, making the fleet harder to manage as a whole.<\/p>\n Declarative infra allows for versioning: Managers can see which stores have which versions of the software. It allows for rolling back of changes if a fix or upgrade doesn\u2019t quite work as planned.<\/p>\n Chick-fil-A has used Git<\/a> as its repository and version control system for the past several years. Chambers said, \u201cGit made a lot of sense for us,\u201d given its cleanly defined API. Since Chick-fil-A uses Suse\u2019s K3 platform<\/a>, its Kubernetes Manifest files are stored in Git.<\/p>\n Every location gets its own repo.<\/p>\n \u201cIt\u2019s fairly simple to build and maintain a solution that has a bunch of Git repos,\u201d Chambers said. It \u201cassigns something location-specific to an agent that lives at the edge that pulls that stuff down and applies changes, and then closes the loop back via an API.\u201d<\/p>\n \u00a0<\/p>\n 4. Secure for a Hostile Environment<\/p>\n Unlike data center computing, your edge device may be in an unfriendly environment, and so you must take extra precautions to guard against breaches.<\/p>\n If necessary, you need a way to kill and erase the contents of a device, through Trusted Platform Module<\/a> (TPM)- or Hardware Security Module<\/a> (HSM)-based devices, Henry suggested.<\/p>\n Most importantly, you have to figure out how to send out the kill signal \u201cwithout making it a vulnerability,\u201d Henry said. This should also be done in a way that doesn\u2019t brick the device itself, unless you want to be traveling to some remote location to install replacement units.<\/p>\n \u00a0<\/p>\n 5. Use Telemetry Sparingly<\/p>\n Telemetry is vital for understanding your remote system, but it can also be a bandwidth hog.<\/p>\n Chick-fil-A had an incident where the credit card payment system stopped working across about 100 stores, Chambers recalled. The culprit was edge applications that were \u201cconsuming all of the available bandwidth,\u201d by just updating their logs, he shared.<\/p>\n So, when designing the telemetry for your remote system, you have to figure out what data you truly need, and what the network constraints are for sending that data.<\/p>\n During routine operations, when everything is running fine, you probably won\u2019t need all the logging data.<\/p>\n Chick-fil-A has a number of apps that run on a store server, as well as a number of edge-connected devices, such as fryers. It collects telemetry data for all of these resources and sends it back to the cloud, where it is dispersed to the appropriate management apps in the cloud.<\/p>\n \u201cOur edge is a platform for many different software engineering teams that build different applications, and so they may have their own preferences about what monitoring tools they use to troubleshoot their app, whether it\u2019s AWS<\/a>\u2018 CloudWatch or Datadog, Grafana or Splunk,\u201d Chambers said.<\/p>\n Use tools that can help deliver the data only when you need it. Datadog<\/a>\u2018s open source Vector<\/a> tool helped Chick-fil-A excerpt data only when it is needed, such as for a short duration of debugging. The app team can, for instance, get all the data for a 30-minute time window and switch it off afterwards.<\/p>\n \u201cWe love our logs in the Air Force,\u201d Henry agreed. \u201cOur logs completely consume things.\u201d<\/p>\n But sending everything back to a data center can be expensive from remote locations.<\/p>\n \u201cYou need to push things back across the wire as minimally as possible, but yet still understand the operation,\u201d Henry said.<\/p>\n Edge devices require multiple forms of storage, Henry advised.<\/p>\n Of course, every edge setup has its own needs, so you need to determine what kind of storage is best, be it block, file or object storage. What is your file system, and what requirements does it have? Ceph<\/a>, for instance, requires multiple disks.<\/p>\n You also need to figure out a disaster recovery plan. You need to answer, \u201cWhat happens when things go wrong?\u201d Henry said. Does the data need to be retrieved, or can you just swap it out for a new unit?<\/p>\n \u00a0<\/p>\n You can enjoy the entire talk here:<\/p>\n<\/p>\n![\"slide](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/16cb2c44-edge-sample.png\")
<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/2299554e-edge-02-1024x574.png\")
<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/fd9db505-edge-03.png\")
<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/975546e8-edge-04.png\")
<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/c4aa9181-edge-05.png\")
<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/ff781a4a-edge-06.png\")
<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/0a22c4b1-edge-07.png\")
<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/6f0aab8a-edge-08.png\")
\n6. Sort Your Storage<\/p>\n![\"\"](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/9221f7b0-edge-09.png\")
<\/p>\n
![](\"https:\/\/www.newsbeep.com\/au\/wp-content\/uploads\/2025\/07\/327440bd-joab-jackson_avatar_1495152980.-600x600.jpeg\"\/)
<\/p>\n