Exceptional operations enable exceptional science

Commanding Integral to observe this unique ‘target of opportunity’ required engineers at ESA’s ESOC control centre in Darmstadt, Germany, to react in exceptionally short time under pressure, which was achieved through excellent teamwork and a great deal of experience with the spacecraft and ground segment from 15 years’ flight operations.

Editor’s note: The news earlier this week was, indeed, exceptional (see “Integral sees blast travelling with gravitational waves”). ESA’s venerable gamma-ray observatory, Integral played a crucial role in this first-ever simultaneous ground-and space-based observation of a cataclysmic gamma-ray burst, which emitted both electromagnetic radiation and gravitational waves across billions of light-years of space.

Integral’s spacecraft operations manager Richard Southworth sent in this report, providing a glimpse behind the scenes of the intense activity at ESA’s ESOC mission control centre in Darmstadt, Germany, that helped make this incredible scientific achievement possible.

Coincidentally, on 17 October, Integral celebrated its 15th year in orbit, demonstrating impressive longevity for a hugely successful mission.

On the afternoon of Thursday, 17 August, Integral was approaching the end of science operations conducted during revolution 1,851 (they are numbered sequentially starting with revolution 1 on launch day in 2002), and, for the Flight Control Team at ESOC, it had not been a smooth day.

The team were in the process of planning routine flight activities for the upcoming orbit 1853, when a new set of urgent, high-priority science observations were received from the Integral Science Operations Team at ESA’s ESAC centre near Madrid; these had to be checked, processed and scheduled for flight operations, replacing the previous, already scheduled observation plan.

Murphy’s Law immediately swung into effect: That afternoon, both the prime and back-up mission planning engineers from our team were unavailable due to leave. While the other engineers on the Integral Flight Control Team are cross-trained to handle mission planning should a short-notice target of opportunity arise, re-planning always takes a little extra care and coordination if you’ve not done it in a while.

Murphy jumped on us again: Around midday a network failure affecting one of the 2 branches of the ESOC operations local area network, occurred, affecting all missions. This meant that we were flying the spacecraft and conducting the urgent replanning using the backup Mission control system. While the prime network was soon available again, capacity was reduced and we continued to conduct all Integral operations from the backup system.

Around 16:00 CEST, I received a call from our rather excited project scientist, Erik Kuulkers, at Estec in the Netherlands, asking whether we could accept and re-plan yet again for an additional target of opportunity that would occur in the very next orbit, due to start shortly after midnight.

Furthermore, he asked if we could urgently extract telemetry data – science information – from Integral that was stored at our Kiruna ground station (due to the aforementioned network problem at ESOC) and forward this directly to the science data centre in Geneva.

I checked and confirmed the availability of one of our engineers to process and schedule the new observation plan, which would have to be done after normal working hours; in addition, our hard-working flight dynamics support team would have to step in for this. 

In my conversation with Erik, I asked him what was so urgent.

He was strangely cagey with his answer, but it soon became apparent when reading the details of this urgent request that it had been passed to ESA by scientists working with the LIGO / VIRGO observatories. 

This could only mean one thing! In short, one of the main aims of our Integral mission was about to be achieved, that being the detection of the electromagnetic gamma-ray signature generated by a cataclysmic event, which had also been seen in the form of a gravitational wave.

All of us on the Flight Control Team got down to work, and time was of the essence.

The target of opportunity request was triggered at 15:58 CEST; by 19:49 CEST, the command files were ready in the mission control system, waiting only for the satellite to rise into view above Kiruna ground station to establish a data link and upload the commands. 

This was a rather fast for Integral re-planning! From Initial request from science to operational files on the control system in less than four hours.

Despite running on the back-up system, execution of the operations was flawless: the commands were uploaded, Integral, our faithful, 15-year-old orbiting observatory reoriented itself in space to point it’s prime instruments at the target, and science data recording began at 06:50 CEST on 18 August.

That marked the end of an intense day but not the end of our story.

The next day, on 18 August, our project scientist received a correction to the position estimate of the target and requested an urgent re-plan of the currently executing observations, an even more complex and challenging planning exercise for our team. 

Nevertheless, planning the new command schedule went smoothly and by mid-afternoon Integral was observing the new target location.

In the next several days, we received two more urgent re-targeting requests in support of this exceptional scientific opportunity. Again these were implemented without incident.

And now the incredible news is out, and everyone working on Integral operations couldn’t be more delighted to have contributed to this once-in-a-lifetime event.

As spacecraft operations manager, I can confirm that this exceptional set of observations were only possible due to:

• A reliable, hot back-up here at ESOC for our mission control system and the computer network that supports us (as all missions at ESOC are so supported). This allowed uninterrupted operations in the face of a significant technical incident.
• The willing and experienced support from our highly motivated and trained Flight Control Team, allowing them to share back-up roles seamlessly even for high-pressure, complex activities.
• A highly flexible operations concept allowing for fast replanning for targets of opportunity.

These factors are part of what makes mission control at ESOC robust, reliable and effective, year-in, year-out.

These take years of patient development, planning, practice and investment to achieve on both the human and technical side, and are part of what makes ESA’s mission control centre one of Europe’s – and the world’s – best.

What better illustration can there be? Fifteen years in orbit constantly evolving, always ready for that one moment, so that even when the fates are against us we can still deliver.