Staying ahead of threat actors in the age of AI | Microsoft Security Blog
www.microsoft.com
Microsoft, in collaboration with OpenAI, is publishing research on emerging threats in the age of AI, focusing on identified activity associated with known threat actors Forest Blizzard, Emerald Sleet, Crimson Sandstorm, and others. The observed activity includes prompt-injections, attempted misuse of large language models (LLM), and fraud.

Summary

This research, conducted by Microsoft and OpenAI, focuses on how nation-state actors and cybercriminals are using large language models (LLMs) in their attacks.

Key findings:

  • Threat actors are exploring LLMs for various tasks: gathering intelligence, developing tools, creating phishing emails, evading detection, and social engineering.
  • No major attacks using LLMs were observed: However, early-stage attempts suggest potential future threats.
  • Several nation-state actors were identified using LLMs: Including Russia, North Korea, Iran, and China.
  • Microsoft and OpenAI are taking action: Disabling accounts associated with malicious activity and improving LLM safeguards.

Specific examples:

  • Russia (Forest Blizzard): Used LLMs to research satellite and radar technologies, and for basic scripting tasks.
  • North Korea (Emerald Sleet): Used LLMs for research on experts and think tanks related to North Korea, phishing email content, and understanding vulnerabilities.
  • Iran (Crimson Sandstorm): Used LLMs for social engineering emails, code snippets, and evading detection techniques.
  • China (Charcoal Typhoon): Used LLMs for tool development, scripting, social engineering, and understanding cybersecurity tools.
  • China (Salmon Typhoon): Used LLMs for exploratory information gathering on various topics, including intelligence agencies, individuals, and cybersecurity matters.

Additional points:

  • The research identified eight LLM-themed TTPs (Tactics, Techniques, and Procedures) for the MITRE ATT&CK® framework to track malicious LLM use.
    • mute@sh.itjust.worksEnglish
      34·
      9 months ago

      Are you saying that Microsoft shouldn’t pour a ton of money into valuable threat research? Are you also suggesting that they shouldn’t have poured billions into security (independently from OS level) in the last 6 years? Or are you suggesting they should exit the security industry completely so for example CrowdStrike can be the undisputed EDR leader that no one can challenge and as such no need to improve due to the lack of competition that the likes of CB or SentinelOne can’t put up?

      When you stopped being edgy just pick one for me please.

      For the rest of you, whatever you think of MSFT, I’d rather they continue pouring billions into security research. Because even if it’s coming from them, it matters….