Ransomware Defence in Depth - It's not just about backup & discovery

So I finally decided I should put some words down about the approach to defending against ransomware attacks. Why you may ask when there’s been so much written about this topic? Basically because so much of the stuff that’s been written has been an opportunity for folks to push their products as the way to save you from this malaise yet they only ever give you half or not even that of a 360 view so encouraging a false sense of security!

I’m not saying this post will give you that 360 view but I’m not going to be pushing anything but yes I will use Google Cloud to illustrate defensive controls . But any large hyper scaler has the suite of products to also be able to give you a more comprehensive way of addressing the threat that is ransomware. I’m just gonna use the cloud I currently know best!

These are my own personal thoughts on the issue for official guidance re this issue go read the government advisory which affects you the most and the cloud hyper scaler posts on the subject!

So let’s start with looking at what the problem is. There have always been bad guys( or gals) out there looking to make a quick buck and causing distress to their victims. However now with everything being connected they have easier access to a wider range of victims both big & small. Size really doesn’t matter to them at all as If they think you can pay you’re a potential target !

So what is ransomware exactly? ( disclaimer this is my definition go look on wikipedia or a gov site for the current prescribed terminology)

Ransomware is code that has infiltrated your systems that can hijack, encrypt , steal data /code etc or introduce unauthorised accounts. This can lead to any one of the following or a combination thereof :

  • Being unable to deliver your service/ Conduct business
  • Losing all access to your systems for a prolonged period of time
  • Being unable to manage your systems for a prolonged period of time
  • Your data being made unavailable as stolen or encrypted or corrupted or deleted
  • Your data being exposed publicly
  • Applications that run your services made unavailable
  • Loss of customers as a result of the above
  • Loss of trust from customers
  • Business has to declare bankruptcy as unable to recover an/or customers take legal actions because of your perceived negligence

Ransomware combines the damage scenarios outlined above with a demand of payment to allow you to recover your systems.

Blackfog has been documenting the state of ransomware and just looking at The state of ransomware in 2021 and The state of ransomware in 2020 reports gives you an idea of the scale of the problem .

You should be aware that things fail all the time and you should be coming from a stance to design for failure . It’s the same with ransomware you need to assume you will be a target and need to design to mitigate for that scenario.

First you need to undertake a full risk analysis to understand the implications of being hit by a ransomware attack.

It’s a business continuity issue that involves not just a standard DR planning exercise but also looking at your security. So following security best practice and having a defence in depth approach is critical . A DR plan gives you a way to recover but you may just be recovering already compromised systems. Many ransomware attacks have been years or months in the planning and by the time that the attack becomes public is well after the systems have been compromised. https://www.cyberark.com/resources/blog/the-anatomy-of-the-solarwinds-attack-chain describes the most recent high profile xample of this infiltrate - wait - attack approach.

Another example of the infiltrate and hang around approach is reported in this fairly scary read APT1: Exposing One of China’s Cyber Espionage Units | Mandiant where it states “Longest time period within which APT1 has continued to access a victim’s network: 4 Years, 10 Months "

So security & DR need to be looked at in tandem as just focusing on one or the other will not help you mitigate attacks. Ransomware is a disaster and you need to plan to recover from that situation . Include recovery from ransomware attacks as part of your business continuity testing plan

The uncomfortable message is that your recovery systems along with not having a tested process for how to act in this situation may be the weak links in the chain.

You need to think about extreme situations how will you be able to operate if your systems are unavailable as part of your business continuity planning? Have you got manual contingency plans? Carry out table top exercises to valiidate and test your processes

Create a threat model to identify the potential areas of attack . Personally I think the following are good places to start with threat modelling https://owasp.org/www-community/Application_Threat_Modeling and https://www.microsoft.com/en-us/securityengineering/sdl/threatmodeling

Understand the controls available to you and identify where you may have gaps

The more in depth your controls you will introduce complexity and costs . So you must have a very prescriptive process as you will likely introduce more errors that will provide ways for the bad guys to get into your systems if you do not. Document the controls you will implement, why you are implementing them, who has the ability to manage them. Assume that even with controls in place you may still get infiltrated.

You will have to balance the risk of everything you have being lost versus the investment needed to defend and what is needed to recover.

Create an incident response plan

If you do not have a Business continuity officer then hire one . They can bring the overall plan together as well as coordinate the various teams from security through to your developers and administrators .

Regularly update and test them . Bob or Alice who may be listed as the go to persons might not be working for the company any more

Plan and implement your defence and recovery plans

As a minimum I would suggest you need to consider the following:

Account management

  • Protecting & isolating accounts
  • Implementing division of duty
  • Least privilege
  • Multi factor authentication methods
  • Break glass processes
  • Admin accounts must not be used for activities such as email etc they need to be restricted to undertaking admin activities

Create isolation zones when configuring your environments

Implement security controls adopting a defence in depth approach

Understanding what is involved with creating a mirror production environment

  • How to avoid propagating potential vulnerabilities via both people and software
  • Time to recreate
  • Introduction of new/ isolated admin accounts in mirror environment

Supply chain security

  • Implement supply chain attack mitigation techniques
  • Know where your source is coming from
  • Montor advisories
  • Also Monitor advisories on up stream software e.g images, third party software, languages used, O/s’s etc
  • Act on advisories

Incorporate security as first class citizens as part of your SDLC process ( e.g secure coding practices, architectural security assessments) this pdf from OWASP is a nice primer

Encrypt basically everything

Manage your encryption keys carefully

  • Monitor keys for usage
  • Monitor keys for public exposure
  • Rotate your keys often
  • Avoid downloading keys to remote machines if you can

Data protection

  • Identify the minimal amount of data required to continue to operate
  • Identify what the minimal amount of raw data is needed to recreate a minimal working environment
  • Store data both in the format you need it and in the raw source form if possible in separate isolated zones away from the active production zone
  • Keep binaries and executable code backed up separately from data
  • Implement DR & recovery mechanisms
    • Have a DR plan in place that is also designed to consider the possibility of a ransomware attack include this in your threat modelling!
    • Implement appropriate security controls for your DR & recovery processes
    • Test test test & more testing of your DR & recovery methods
    • How far back can you realistically rebuild from? Ransomware attacks can be lying dormant for some time before the attack is initiated. Where do you keep that far back restoration capability

Implement logging, monitoring and auditing

Train your staff in process and procedures to help prevent social attacks and to try and prevent behaviours that can lead to attacks i.e don’t click on suspicious links, how to contact your security team etc etc

It maybe tempting but you really should not pay any ransom demands. You’ve all seen the movies that does not end well. Law enforcement agencies around the global are advising not to pay these demands. The UK national cyber security centre states “Law enforcement do not encourage, endorse, nor condone the payment of ransom demands” with other equally good guidance on mitigating malware & ransomware attacks here . Spend that money on resilience and recovery instead!

The US department of treasury has an advisory that states that companies that facilitate ransomware payments to cyber actors on behalf of victims, including financial institutions, cyber insurance firms, and companies involved in digital forensics and incident response, not only encourage future ransomware payment demands but also may risk violating OFAC regulations.”

You can find plenty of other advisories & guidance from other countries .

As a minimum review and act upon governmental guidance ( I linked through to the UK NCSC guidance above as an example)

That’s a lot to take into consideration and that was with me just listing out what I consider to be the minimum . As I mentioned at the beginning all Cloud hyperscalers can help you in some way with inbuilt mechanisms but I’m just going to end this post with listing out some Google cloud controls that you can use to help you with implementing a defence in depth approach to defending against malware and ransomware attacks. This is not a comprehensive list . This post is long enough as it is!

Google cloud DR planning - The series of Disaster recovery docs help you to understand the cloud native way to implementing a disaster recovery plan on GCP .

Resource hierarchy This provides you with the ability to have different projects that have different access controls set in terms of what accounts can access what and what access rights they have there. The projects can be grouped under different folders. In an ideal world separate organisations may at first seem the way to achieve the best degree of separation but the overhead of managing those may introduce a degree of complexity such that it offsets the perceived protection. Well controlled divisions via folders and projects, using SC VPC ( see below) and IAM you can achieve a high degree of defence in depth

Identity and access management (IAM ) lets you manage access control by defining who (identity) has what access (role) for which resource. It helps you to implement least privilege

Operations: Cloud Monitoring & Logging provides you with a way to implement logging, monitoring and alerts. But don’t just watch them, you need to have processes in place to act on the signals you are receiving.

Security command center This helps you to detect threats against your google cloud environment. It also provides a way to identify security misconfigurations as well as being a central place to discovery and view your assets. You can also integrate with many third party security solutions

Data encryption - Google cloud has a number of white papers discussing encryption at rest and in transit you should understand these but more importantly you need to understand what options you have to have more control for individual products. The Envelope encryption | Cloud KMS Documentation is a nice primer

Also see data encryption options for cloud storage Encrypt disks with customer-supplied encryption keys

Service accounts are an important resource when working with Google cloud so follow best practices when working with them see Best practices for using and managing service accounts and Best practices for securing service accounts | Cloud IAM Documentation and if you really must download keys make sure you have key rotation process in place . Understand the mitigations third parties have in place such as Github Secret scanning which warns you about exposed secrets . Maybe you could avoid using external repositories and put processes in place to prevent their use. GCP provides private git repositories for this use case.

You can also put in place preventive measures to stop keys from being committed to your git repo. One open-source tool you can use is git-secrets . This is configured as a git hook when installed

Use Secret management solutions like Hashicorp vault and Secret manager ensuring least privilege is applied and you rotate your secrets regularly

VPC Service Controls allows you to create perimeters to help mitigate against data exfiltration

Software Supply chain mitigations Binary Authorization , require images to be signed by trusted authorities during the development process and then enforce signature validation when deploying.

Shift left so implement security earlier in your ci/cd process Help secure software supply chains on Google Kubernetes Engine is a nice example of how to do this

Patching and scanning . You should always patch as in many cases the attack vector is by exploiting a vulnerability that has a patch available but you haven’t got around to patching! Google cloud has services to help with this - OS patch management | Compute Engine Documentation Getting vulnerabilities and metadata for images | Container Registry

Trust nothing inside or out this is zero trust. Verify everything Google practices zero trust and have built upon this to provide services such as beyond corp Traffic Director integrates with CA Service are ways that zero trust approaches are integrated with Google cloud services.

Priyanka’s comic on zero trust know your devices is a nice high level explanation. UK NSCC thinks zero trust i s a good idea too .

Multi factor authentication is a key defence against account hijacking and Google Titan Security Key are worth investing in .

I could go on but I think this gives a good idea of the sorts of controls you should expect of the Cloud hyper scaler of your choice to be able to offer you in your efforts in protecting against ransomware attacks. There may not be direct equivalences of the controls I have listed for google Cloud but you should be looking for controls that address the threats you identified when creating your threat model.

Thanks are owed to one of my colleagues for reviewing this despite its length ! You know who you are!