Advanced Cyber Incident

Update: 27th Jan 2023

Situation Overview

Staffplan product update issued under the affected product tab

Latest product updates

Update: 20th Jan 2023

Situation Overview

Staffplan product update issued under the affected product tab

Latest product updates

Update: 16th Jan 2023

Situation Overview

Staffplan product update issued under the affected product tab

Latest product updates

Update: 4th Jan 2023

Situation Overview

Staffplan product update issued under the affected product tab

Latest product updates

Update: 2nd Nov 2022

Situation Overview

We are currently working on a proof of concept with Citrix (a secure multi-factor authentication gateway) to enable us to provide Caresys in a secure way. We aim to complete the proof of concept and internal testing by mid-November.

Once the internal testing is complete we will onboard Customer #1 quickly afterwards with the aim to bring all customers back on the core Caresys system by the end of December.

A member of our team will contact you and provide a fact sheet and a list of your users. They will talk you through the onboarding process and ensure they can answer as many questions as possible before your switch-on date.

Latest product updates

Update: 17th Oct 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 13th Oct 2022

Situation Overview

As of: 30th Sept 2022

Below please find information summarising Advanced’s current understanding of the recent cybersecurity incident and the actions the Company has taken and continues to take in response.

Attack Path

The earliest evidence of threat actor activity identified on the Advanced network was on 2 August 2022 and the most recent date of activity is 4 August 2022.The threat actor initially accessed the Advanced network using legitimate third-party credentials to establish a remote desktop (RDP) session to the Staffplan Citrix server. During the initial logon session, the attacker moved laterally in Advanced’s Health and Care environment and escalated privileges, enabling them to conduct reconnaissance, and deploy encryption malware. Immediately prior to encrypting systems, the threat actor copied and exfiltrated a limited amount of data.

Our threat intelligence and forensic firms have confirmed that the malware strain used in this attack was LockBit 3.0. We are happy to share additional Indicators of Compromise (IOCs) with Advanced customers upon request.

The forensics are very nearly completed and at this stage, it is highly unlikely there will be additional findings. We expect to have a formal forensic report completed in the coming weeks, which will be available upon request to our customers.

Containment

Upon first detecting suspicious activity, our security team promptly disconnected the entire Health and Care environment to contain the threat and limit encryption to a small number of systems. This action also prevented any further threat actor activity within the environment. However, by taking this action, our customers lost access to Health and Care platforms, as well as a limited number of non-health and care environments and services, such as eFinancials.

Remediation

Once our teams were able to contain the threat, we promptly began rebuilding and restoring impacted products and systems in a separate, secure, and new environment. We also implemented the below immediate measures to the Health & Care environment:
• Scanned for identified Indicators of Compromise (IOCs)
• Installed real-time monitoring, detection, and response agents
• Reset passwords
• Rebuilt and hardened compromised systems, including Domain Controllers
• Enhanced network segmentation
• Strengthened firewall rules

These are only some of the efforts we’ve made to enhance our cybersecurity defenses and we are continuing to evaluate additional steps we can take to further secure our environment.

Recovery

Our teams have worked around the clock to recover from this attack as quickly and safely as possible.

Although we were equipped and able to completely rebuild certain health and care products by the Monday following the incident, we were required to satisfy an assurance process set forth by our partners at the NCSC, NHS, and NHS Digital. This assurance process helped to provide confidence that once our rebuilt products were ready to go live, they were fully remediated and safe for our customers to use. As we learned more about this assurance process and adjusted in real time to meet certain requirements, it took longer than expected, which has impacted our overall recovery timeline. We have prioritized safety and security during every step of our recovery process.

Our Health and Care and environments beyond Adastra and 111 will also require additional compliance checks, scanning, and going through the same assurance processes. This is time consuming and resource intensive and it continues to contribute to our recovery timeline. As we work through scanning and clearing systems, we are in parallel continuing to assess and/or develop recovery plans for remaining impacted products.

We are working diligently and bringing all resources to bear, including outside recovery specialists, to help us restore services to our customers as quickly as possible, and in the interim, providing data extracts and assisting with contingency planning as appropriate.

Data Review

We can confirm that the perpetrators of the attack, who were financially motivated in nature, were able to temporarily obtain a limited amount of information from our environment pertaining to approximately 16 of our Staffplan and Caresys customers. We have now notified each of those affected customers as the controllers of the exfiltrated data.

Importantly, no data was taken from other products.

Additionally, we were able to recover the limited amount of data obtained from our systems and we believe the likelihood of harm to individuals is low. This is based on our expert threat intelligence vendor's considerable experience with cases of this nature and the fact that there is no evidence to suggest that the data in question exists elsewhere outside our control. We are, however, monitoring the dark web as a belt and braces measure and will let you know immediately in the unlikely event that this position changes.

We have been and continue to be in contact with the ICO, the NHS, the National Cybersecurity Centre (NCSC), and the National Crime Agency to provide regular status updates on this incident.

Again, Advanced has now given required notice to all affected data controllers. If you were not contacted, your data was not copied out of the environment.

Latest product updates

Update: 29th Sept 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 22nd Sept 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 13th Sept 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 2nd Sept 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 25th Aug 2022

Over the last three weeks we have been focussed on assessing our ability to restore and provide reconnection to these services. Due to a number of factors, this has been more complex than we initially anticipated.

Staffplan

For hosted Staffplan customers, we envisage that contingency measures could be required for a further four to six weeks. 

For our Staffplan customers we have been able to make data extracts available to assist organisations in their day-to-day operations.  Customers who haven’t already requested their data can do so by submitting a ticket to their Customer Support Team.

Caresys

For hosted Caresys customers, we envisage that contingency measures could be required for a further six to eight weeks. We anticipate data to be available within the next two weeks.

Crosscare

For hosted Crosscare customers, we envisage that contingency measures could be required for a further eight to twelve weeks.  We anticipate data to be available in the next two to three weeks.

eFinancials

For our NHS Trusts using eFinancials we have now satisfied the assurance criteria set down by the NHS and NCSC and customers are now reconnecting via the HSCN network which was unaffected by the incident, but taken offline as a precaution.

All our efforts are focussed on restoring services for our customers as quickly as we can, whilst ensuring we do so in the most secure way possible.

Latest product updates

Update: 23rd Aug 2022

Recovery Update

The NHS England EPRR incident management team has now communicated its findings based on evidence from our internal security assurance activity. NHS 111 customers are therefore beginning to reconnect to Adastra in line with the process set out by NHS England. At present we have a number of customers who have successfully reconnected [including the London and South Central Ambulance Services] with Adastra OOH providers following. We continue to prioritise safety and security in all of our decision making and are approaching this restoration process with diligence and rigour.

Situation recap – cybersecurity incident

Advanced experienced a disruption to our systems that we have since determined to be the result of a cybersecurity incident caused by ransomware. In response, we immediately took action to mitigate any further risk and isolated all of our Health and Care environments, where the incident was detected.

The customer groups impacted either directly or indirectly were Adastra, Caresys, Odyssey, Carenotes, Crosscare, Staffplan and eFinancials. All other products were unaffected by this incident.

In the time since the attack, we have progressed our investigation into the incident, including recovering and restoring affected systems.

Forensic Investigation

Our forensic investigation is progressing in line with our timeline and plan. We are now building a much clearer picture of the incident. In parallel, our third-party experts are well advanced in their investigation into any potential data impact as a result of the incident. We will update customers as appropriate and comply with any applicable notification obligations.

Latest product updates

Update: 19th Aug 2022

Situation Overview – cybersecurity incident

As you know, Advanced experienced a disruption to our systems that we have since determined to be the result of a cybersecurity incident caused by ransomware. In response, we immediately took action to mitigate any further risk and isolated all of our Health and Care environments, where the incident was detected.

The customer groups impacted either directly or indirectly were Adastra, Caresys, Odyssey, Carenotes, Crosscare, Staffplan and eFinancials. All other products were unaffected by this incident.

In the time since the attack, we have progressed our investigation into the incident, including recovering and restoring the affected systems.

Recovery Update

For NHS 111 customers using Adastra we have now completed our internal security assurance activity and will be sharing evidence with NHS and NCSC for review. Once the evidence provides a high confidence level this will be communicated through via the NHS England EPRR incident management team and customers will be able to reconnect in line with the process set out by NHS England. Monday next week we will be moving forward with the phased process of bringing these organisations back online. The order in which providers reconnect to Adastra is being set by the NHS England EPRR Incident Management Team, which has communicated its process to all relevant users.  For our Adastra OOH Customers, we will be working to share a more detailed recovery plan next week.

The completion of the Adastra assurance process also enables us to move forward with re-connecting our eFinancials customers via the HSCN network, which was unaffected by the incident, but taken offline as a precaution. Our leading third-party forensic partners, including Mandiant and the Microsoft DART team, have been running tests on nearly 150 eFinancials servers to ensure they can be brought back online safely and our customers who use them can feel confident in reconnecting once service is restored. We remain in constructive dialogue with the NCSC and other government departments to ensure a smooth transition once customers are able to re-establish connection. NHS organisations would need to follow the guidance set out by NHS England before reconnecting. We expect this to be possible from today onwards and will be sharing details of how to initiate this reconnection with each customer directly.

We also continue to move forward with recovery planning efforts for our other directly impacted customers.

For Staffplan customers we have been able to make data extracts available to assist organisations in their day-to-day operations. Data available now includes care worker details, service user details, carer roster information, service user contacts, and service user schedules. These datasets can be obtained by submitting a ticket to your Customer Support Team. We continue to seek additional workarounds for how we can make this data available to customers, and will provide updates via our website as usual.

Most of our Carenotes customers now have access to their log shipping data. For those who don’t we are continuing to contact them on a 1:1 basis to discuss data requirements. There is further information available on our Customer Support Site for customers who want to capture clinical notes data.

For our Caresys and Crosscare customers we are still working through our technical assessment to determine the next steps towards recovery.

While our recovery work progresses, we thank customers for continuing to implement their contingency measures. We will provide regular, service-specific updates on our website portal as our efforts progress, and hope to be in a position to provide more concrete news on timelines by the end of next week.  

Forensic Investigation

Our forensic investigation is progressing in line with our timeline and plan. We are now building a much clearer picture of the incident’s root-cause and will soon be in a position to confirm and share Indicators of Compromise (IOCs) with customers on request. In parallel, our third-party experts are well advanced in their investigation into any potential data impact as a result of the incident. We will update customers as appropriate and comply with any applicable notification obligations.

We recognise that this has been a challenging time for our customers, and we appreciate your patience and understanding as we work to recover from this attack. We continue to prioritise the safety and security in all of our decision making and are approaching this restoration process with diligence and rigour

Product updates also issued under the affected products tabs.

Latest product updates

Update: 15th Aug 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 12th Aug 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 11th Aug 2022

Situation Overview

Product updates issued under the affected products tabs.

Latest product updates

Update: 10th Aug 2022

Situation Overview

As you know, Advanced recently experienced a disruption to our systems that we have since determined to be the result of a cybersecurity incident caused by ransomware. On August 4, 2022, at approximately 7 am, our teams identified the cybersecurity incident. In response, we immediately took action to mitigate any further risk and isolated all of our Health and Care environments, where the incident was detected.

The customer groups impacted either directly or indirectly are Adastra, Caresys, Odyssey, Carenotes, Crosscare, Staffplan and eFinancials. All other products are unaffected.

Response and Containment

We moved swiftly to engage leading third-party forensic partners including Mandiant and the Microsoft DART teams to conduct an investigation and help to ensure that our systems are brought back online securely with enhanced protections. Moreover, we remain in contact with the NHS, NCSC, and other governmental entities and are providing them with regular status updates. We have also been in contact with the ICO and will continue to be responsive to any questions they may have.

We want to stress that there is nothing to suggest that our customers are at risk of malware spread and believe that early intervention from our Incident Response Team contained this issue to a small number of servers.

Since our Health and Care systems were isolated at the end of last week, no further issues have been detected and our security monitoring continues to confirm that the incident is contained, allowing our recovery activities to move forward.

Remediation and Recovery

We are rebuilding and restoring impacted systems in a separate and secure environment. To help all customers feel confident in reconnecting to our products once service is restored, we have implemented a defined process by which all environments will be systematically checked prior to securely bringing them online. This process includes:

• Implementing additional blocking rules and further restricting privileged accounts for Advanced staff;
• Scanning all impacted systems and ensuring they are fully patched;
• Resetting credentials;
• Deploying additional endpoint detection and response agents and;
• Conducting 24/7 monitoring.

Once these measures have been taken, we will bring environments online and assist customers in reconnecting safely and securely as part of a phased return to service.

With respect to the NHS, we are working with them and the NCSC to validate the additional steps we have taken, at which point the NHS will begin to bring its services back online. For NHS 111 and other urgent care customers using Adastra and NHS Trusts using eFinancials, we anticipate this phased process to begin within the next few days. For other NHS customers and Care organisations our current view is that it will be necessary to maintain existing contingency plans for at least three to four more weeks. We are working tirelessly to bring this timeline forward, and while we are hopeful to do so, we want our customers to be prepared. We will continue to provide updates as we make progress.

Forensic Investigation

As you can imagine, we are in the early stages of our investigation into this incident and are working alongside our third-party forensic partners to gather more detail. While we have not yet confirmed the root cause – and this may take time – please rest assured we will keep you updated as we learn more. We are also in the process of confirming Indicators of Compromise (IOCs) and will share those with our customers when they are available.

With respect to potentially impacted data, our investigation is underway, and when we have more information about potential data access or exfiltration, we will update customers as appropriate.  Additionally, we will comply with applicable notification obligations.

We want to thank our customers for their continued patience and appreciate the support and understanding that they and our government partners have shown to us as we progress our multi-phased response. We fully understand the challenges this incident has caused for many of our stakeholders and will continue to provide additional updates as soon as we have more relevant information to share.

Update: 5th Aug 2022

Health & Care Hosted Infrastructure Outage – Update & Recovery Plan

Firstly, we would like to thank you for your patience and understanding whilst we have worked through the investigation of a security issue identified early morning on 4 August 2022.

What has happened?

A security issue was identified, which resulted in loss of service on infrastructure hosting products used by our Health & Care customers:

- Those products identified as being directly affected and whose servers and
network connections were taken immediately offline are Adastra, Caresys, Odyssey, Carenotes, Crosscare and Staffplan.

- Those products indirectly affected but who have lost connection to their systems due to precautionary taking down of the HSCN network and Internet facing connections are eFinancials. The incident is related to a cyber-attack which can be pervasive in nature and so, as a precaution, we immediately isolated all of our Health and Care environments. This occurred at approximately 7am on 4 August 2022 when, to mitigate any further risk of impact to our customers critical systems, we took all Health & Care systems offline. Since then, no further issues have been detected although we appreciate you will not have been able to access your Advanced services.
What we can reassure you about is that the attack is contained and not spreading. We continue to investigate and although we have found no evidence that any personal data has been compromised, we are providing notice to you, as we are required to do as Data Processor, about the incident.

What proportion of the Advanced infrastructure has been impacted?

This incident relates to those customer groups mentioned above. Early intervention from our Incident Response Team has contained this issue to a small number of servers representing 2% of our Health & Care infrastructure limiting the impact. The protection of your services and data is paramount in the actions we are taking.

When will this be resolved and services recovered?

Currently, for our customers who were directly affected you will need to maintain your contingency measures in place for the duration of the weekend and into early next week. We appreciate the inconvenience this will cause to you and your teams. For our customers who were indirectly impacted but who rely on the HSCN network we are in the process of restoring connectivity today and will keep you updated on this. Our full focus is on restoring services as soon as is possible, but not until we are confident that appropriate protection measures are in place and that it is safe to do so.

Why hasn’t Advanced communicated with us?

During yesterday we provided two updates to the Customer Support Portal. We realise from the incoming enquires we have received that this wasn’t enough. We apologise for this. Our internal security team along with key technology and security partners were working throughout the day to understand the impact and implications of the attack as part of our priority incident response. Now that we have a fuller understanding of the attack and impact we can focus on recovery. We are committed to providing much better levels of communication to you from here on in.

What should we do?

We ask that you continue to implement your contingency measures and internally ensure your own antivirus and other security services are up to date. We are working closely with our NHS and other heath and care bodies, technology and security partners and will continue to provide incident updates to you via email and on our customer support sites as new information becomes available. Over the last two years, Advanced has invested significantly in our Infrastructure Resilience with the primary focus being on Service Continuity and maintaining our ISO 27001 compliance and annual certification ensuring our systems and processes are up to date and meet the demanding standards of it. If you have any further questions, you can contact your Support Team in the usual way.

If you have security or data questions, please contact our Data & Security Team dataprotection@oneadvanced.com.

We appreciate your continued understanding and patience.