Editor’s Note: Satellites taking photos of other satellites requires satellites with decent cameras, a lot of computational smarts – and until a few years ago in the U.S., security clearance. Non-Earth Imaging (NEI) came into the Space Domain Awareness (SDA) public realm because it’s never been restricted elsewhere. William Crowe, Founder and CEO of Australian NEI pioneer HEO, walks us through the technical and bureaucratic challenges of bringing NEI to market.
OT: HEO took its 4,000th image a few months ago. How did it get started?
William Crowe: We started as an asteroid mining company, and the insight that drove the company was that there are all these asteroids coming super close to Earth. So you don’t actually need to leave Earth orbit to investigate and do mineral prospecting on a lot of asteroids. And the reason why you can choose any old asteroid rather than going to a specific one is that actually the mapping between meteorite types and asteroid types is not complete. So you need a vast array of random asteroids and there’s enough coming though the Earth-Moon system that you can just go out and image those. So, that was the insight.
Sadly, we didn’t realize that there’s not really a business there yet. So, the insight didn’t quite work out in a business sense, but the ability to image things relatively cost effectively turns out to be quite a transferable business case. The primary way of looking at asteroids is to do fly-by imaging. You fly past them on the way to a planet, typically, and you’ll take these great images.
Around Mars, too, the Mars orbiters were doing space images of each other and capsules as they landed as well. So, it’s interesting that this wasn’t a secret to the scientific community, but you couldn’t find an image online when we started. There was one that we found of MVAT taken from a French satellite.
It was prohibited to even say the names of what used to be called Sat-squared.
So we thought that this appears to be not crazy. The scientists didn’t have any blockers. But in the U.S., they did. It was prohibited to even say the names of what used to be called Sat-squared. You couldn’t say that outside of a SCIF [Sensitive Compartmented Information Facility – ed.]. We later found out that the Internet was mainly scrubbed clean except for these few examples mainly of Mars but one of an Earth-orbiting satellite. There were heaps of photos of the International Space Station, heaps of photos of rendezvousing spacecraft.
We thought that if that’s possible, then surely you can do it at a higher speed. We started testing this first with Landsat. It turns out that Landsat’s accidentally capturing images of other spacecraft all the time. Like, it’s just always turned on over land and sometimes a spacecraft will fly below it. So, we’re able to test our software that way. And then the insight was actually monetizable, that we could use Earth Observation satellites to do this. It turns out that they’ve got a lot of downtime. More than 95% of the time, optical Earth Observation satellites can’t take images of the Earth. Either it’s cloudy or over the ocean or the ice caps or whatever, or it’s nighttime. And so we can use these gaps in its imaging to take these images and we can provide a cash source that the companies running these sensors didn’t have access to before.
So, the real insight is getting access to these, and using software primarily with existing satellites and then being able to transform them to image other spacecraft as they fly by. And then there’s a lot of smarts in order to figure that out. But once you have these building blocks you can start to build that one up.
OT: How long did it take to go from the point of saying, “Okay, we can do this, “ to the first image?
William Crowe: It took about six months. It was pretty quick because we were building Astro for the first few years. So from 2016 til 2019 essentially, and in 2019-2020, we were building the capability. It happened in a relatively short amount of time, but not having to build our own satellite and using other people’s satellites was the key to making it work. That’s actually a really cool thing because repurposing satellites, it’s not a common phrase that you find in the world.
We started just showing these images that we’d taken in these exercises and I think that’s’ when they freaked out.
OT: So, you have image number one, and did the Americans start to freak immediately, or how long did it take for them to find out about you?
William Crowe: Well, we approached them. There were these events called SACT. And they were doing space domain awareness exercises; looking at objects in space and tracking them, it was really difficult for the U.S. government to track all the objects on their own. There had been an increase in the number of objects and that was around the time that Starlink really started to launch a lot and OneWeb had already launched a lot of satellites. So, they knew that the game was changing, and they had these exercises. Space Force’s predecessor held them, and they encouraged commercial companies to join in. So, we did and we started just showing these images that we’d taken in these exercises and I think that’s’ when they freaked out.
It happened because they have people on what they call the high side, people with clearances, etc. And there’s another kind of wargaming exercise happening at the same time on this high side. They essentially have a one way glass so these people can look at what everyone in Commercial is doing. I think there were some very surprised expressions, like, “What’s this Australian company doing?” By the way, at this point, we still had no idea that this was done.
We were like, “Oh, we’re providing this brand new capability and no one knows it exists. And frankly, the majority of other people who we could see in this exercise had no idea. It was to the point where I think it was so alien, they had no idea how they could place this technology.
They were like, “Yeah, we should know where an object is.” And we were saying, “You should also know what an object is” They were saying, “Great. Well, you know, we can look at this point of light and it will change magnitude and from that we can infer all these things.” We’re like, “Yeah, you can do that, which is really hard and doesn’t really give you a close answer, or you could just take a photo of the satellite, which is what we’re doing.”
I don’t know if you’ve seen the Jedi curve, and the midwit meme, but it’s kind of like everyone was in the middle, saying,”Oh, you can do all these things and you can figure all this stuff out,” and we were like, “or you could just do the simple thing and get the image and see exactly what its purpose is.”
So they were watching through the one-way glass and saw that this was happening. They were excited to meet us after that and from participating in that and showing actual images to meeting them was less than a year process.
OT: Have you found much in the way of other commercial companies starting to say, “Hey wait a minute! We have connections with Landsat.”?
William Crowe: We’ve seen some commercial companies say that, but overall they’re very slow. What I’ve seen in the commercial space and am still trying to get my head around is why this is, but I think people are quite often locked in to very specific use cases and just building more and more complex stuff and having these long lead times. It’s probably because of the way the procurement programmes are happening, and mainly government work with long lead times and money specifically to build new stuff as well. So I think there was a lot of cultural inertia that was building up whereas someone like us with no baggage could just come along and do the thing immediately. So yeah, there was a lot of cultural inertia.
By the way, I saw that there was another researcher earlier than us who had looked at Landsat and seen another object. So, it was interesting to see that at the end of the day, we’re replicating their work. So, this was known but it was so hidden in a torrent of other academic work, and the person who wrote it had no idea that it had these implications. I think the government was very happy to let that die a death in a journal somewhere. It makes you think about how many other things are just sitting out there. Someone’s thought of it, but not really thought to progress beyond that state.
OT: By the time they get to you, you’re well past your first image, but what about the improvements in image quality? Was that a matter of algorithm improvements, or was there more in the process?
William Crowe: A lot of it is algorithmic. Part of it is using the Sun as the light source, so you optimize the angle to get the best image. Then there’s other things like how do you deblur? How do you get the satellite to move or how do you deblur post-processing and then there’s a bunch of other things as well. But really, the key to this is how do you either get a satellite with a higher resolution camera or how do you get access to more satellites to increase the chance that it passes relatively close to the object you care about because the resolution’s kind of proportional to the distance away that you are. Having a network of these sensors is really important because at the end of the day, no matter how great your sensor is, there will be an object too far away for it to image well.
What you should do is have as many sensors that you have access to as possible. And then, additionally try and have the average of those sensors to be from relatively high resolution cameras as well. You put those together and then we’re building a network from there.
That’s why it’s important for us to work with as many Earth Observation satellite companies as possible, because the thing about Earth Observation satellites is that each of them is in a different kind of shell. They each want all their satellites to be an equal distance from the Earth so they get a consistent image. Fortunately, each of those is designed for a slightly different altitude.
So, if we can group all those together, then the likelihood that each is going past an object we care about increases dramatically because typically, satellites are in circular orbits and each of these is at a different shell. You want their shell to be as close to one of your shells as possible. Having as many shells as possible is really important. And the other thing that helps is having slightly different inclinations, although I would say that altitude is the most important thing.
OT: The first thing that people think of regarding image quality, it’s got to be your AI improvements, but no, there’s actually more more that goes into it.
William Crowe: You can do AI, but you’ll probably mix stuff up. So, it’s better not to use it too much. You can use it, but not too much.
OT: Back to regulatory: Are there any regulatory hurdles that are impeding your work these days?
William Crowe: Backtracking a little bit, we didn’t know this when we started, but in Australia there was no regulatory regime preventing this from happening. There was. So, that’s the first thing. Then the partners that we partnered with, were also outside the U.S., so again, no regulatory concerns. But in the U.S., they first had regulations preventing this, and additionally, there was this veil of secrecy. You couldn’t even find out that this was happening until you built up to that classification level. So those two things were happening. In Australia, we have good rocket legislation and some payload building legislation, but that’s it. It’s a huge land mass, and additionally, there’s national telecommunications carriers. So, essentially the regulation was built around those two whereas there is no Earth Observation capacity to this day.
We’ve just purchased what’s Australia’s first ever commercial remote sensing satellite. It’s taken until 2025, as we bought it in the last week of the year. There’s no regulatory regime around that and we’re able to exploit that without realizing that it’s exploitable. We just started doing it, and then there was a declassification event in the U.S.
Now, you can look at the earliest papers about sat-sat imagery. You can actually see those now. The first stuff started happening in the ‘70s with the Americans looking at Russian satellites. And then there was a deregulation event as well. They have this remote sensing ruling where there’s three levels and the third only exists in the U.S. and only U.S. secret organizations can get access to that and that’s where non-earth imagery was. There’s also number one, which is, “this is done so much across the world so therefore pretty much anyone can do it, so we shouldn’t prevent anyone in the U.S. from doing that.”
Us existing allowed it to go from level three to one in a very short period. That was great. What we have been told by some people in the U.S. is that if we were a U.S. company, because we’re still really the only company doing this at scale outside of the U.S., and even in the U.S. there aren’t many of these companies. If we were to redomicile to the U.S., there’d be the potential for them to attempt to reregulate it. If we were a U.S. company, and these other companies are U.S. companies, they’d make it tier three again.
Because of that, we remain Australian headquartered even though we’ve been through Y Combinator, which asks all of its companies to become U.S. companies. We have had exclusions from that because of this regulatory barrier and you’ve seen companies like ICEYE with exactly the same problem. So, there’s a few companies like ours allowing the regulatory gates to stay open. Weirdly, we’re helping our competitors, because if we were to redomicile to the U.S., then suddenly, everything might go back in the box. So, it’s important that we stay here today. As far as the EU or Japan, there’s no problem.
OT: Was this regulatory threat one of the reasons why you bought the satellite?
William Crowe: No, not really. It was primarily so we could do more advanced testing and do it really fast. It took a really long time for our partners to build up trust that our way of doing things wasn’t going to break their satellite and prevent it from doing its main mission, which is earth observation.
We’ve been implementing new imaging modes, but it’s really slow because we want to be really careful, make sure that this doesn’t impact the main operations. Now that we own a satellite, we don’t have to worry about those concerns and that allows us to test more rapidly, then potentially roll those new and improved modalities back to the rest of the satellites that we work with. So, it’s about experimenting with the technical capability and the business model that we use as well. So, it’s really important that we do that.
I would say that in a weird way, it’s actually unfair to call it lax. I would say that because there’s new ground being broken, it allows regulations to be built around the technology of today rather than the technology of five years ago. And that was the state of affairs in the U.S.
I think that’s really beneficial and it allows for these new capabilities to be tested and I think most people don’t know how fast space is moving. It’s really exciting but it means that we need the right kind of rails to help and I would say in the old space fairing nations those rules aren’t fit for purpose at this point. So, yeah, really it helps us to do that and has this more beneficial regulatory environment as well.
And when it comes to ‘sovereign’, though, who knows what that means? There will be a degree of sovereignty, What I found, though, is that everyone has a slightly different definition of sovereignty. Some people will be very happy and some people will be like, “actually, it’s not quite enough yet, although a good direction that you’re moving in”, right? Because it’s not owned by the government it’s not really sovereign, Somebody will have to say that. They’re like, “Yeah, all the components weren’t built here.” Just how much do they want?