Physics Nobel scooped by machine-learning pioneers
- Neural networks and physical systems with emergent collective computational abilities. (Classic 1982 paper) - J.J Hopfield
Open Source, security and social media plus random digital things.
Physics Nobel scooped by machine-learning pioneers
Microsoft's AI Red Team has published a new paper titled “PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI Systems” on arXiv.
Generative AI (GenAI) has increased in popularity over the past few years, since applications such as ChatGPT captured the zeitgeist of the new wave of GenAI developments. This disruptive and highly innovative technology has become more widespread and more easily accessible than ever before. The increased capabilities of these models have inspired the community to incorporate them into almost every domain, from healthcare [21] to finance [4] to defense [22]. However, with these advances comes a new landscape for risk and harm. GenAI models are generally trained on huge datasets scraped from the Internet [10], and as such the models contain all the potentially harmful information available there, such as how to build a bioweapon, as well as all the biases, hate speech, violent content, etc. contained in these datasets [20]. When a company releases a product that uses GenAI, it inadvertently contains these potentially harmful capabilities and behaviors as an innate part of the model. As with any rapidly advancing technology, the development of new tools and frameworks is crucial to manage and mitigate the associated risks. Generative AI systems in particular present unique challenges that require innovative approaches to security and risk management. Traditional red teaming methods are insufficient for the probabilistic nature and diverse architectures of these systems. Additionally, although there is a promising ecosystem of existing open-source GenAI tools, there is a dearth of tools grounded in practical application of GenAI red teaming.
A. Gandalf
To demonstrate the effectiveness of the attacker bot mode, we conducted a proof of concept using the chatbot Gandalf from Lakera [12]. Gandalf serves as an effective test bed for evaluating the capabilities and flexibility of the PyRIT framework. Designed to help users practice crafting prompts that can extract a password from the chatbot across ten progressively more difficult levels, Gandalf introduces additional countermeasures at each level, including stronger system prompts, block-lists, and input/output guards. To evaluate the effectiveness of the Red Team Orchestrator in PyRIT, we developed targets and scorers tailored to Gandalf. The experimental setup involved configuring the following components within PyRIT: 1) Target Endpoint: Gandalf was set as the target LLM. 2) Red Team Bot: GPT-4o was the LLM powering the red team bot. 3) Attack Strategy: A text description of the objective for the red team bot. In this case, the objective is to extract the password from the Gandalf (the target endpoint). 4) Scorers: Custom scoring engines were implemented to evaluate the responses generated by Gandalf. We used the red team orchestrator to probe Gandalf and extract the passwords for Levels 1-4. PyRIT successfully extracted the passwords by leveraging its self-reasoning capabilities, which keep track of conversation history to increase the likelihood of success in subsequent prompts
PyRIT (Python Risk Identification Tool for generative AI
In November 2023, Microsoft introduced the Secure Future Initiative (SFI) to enhance cybersecurity protection for Microsoft, its customers and the wider industry.
For an update on the progress of the SFI, it is recommended to review the SFI Progress Report from September 2024.
Co-chaired by the BEUC - The European Consumer Organisation and the EACB, a multi-stakeholder group composed of associations of banks, payment services providers, their clients and several public authorities, under the auspices of the Euro Retail Payments Board, produced a Report with recommendations on how to tackle on fraud related to retail payment.
🔑 The report identifies four "gamechangers" for effective fraud prevention and mitigation:
➡ Cross-sectoral collaboration and shared responsibilities beyond the payment industry.
➡ Sharing fraud insights and data across sectors.
➡ Supervisory enforcement and cooperation at the EU level.
➡ Product design that prioritizes consumer protection
You can download it here: https://www.ecb.europa.eu/paym/groups/erpb/shared/pdf/21st-ERPB-meeting/Report_from_the_ERPB_Working_Group_on_fraud_prevention.pdf
Tracking the historical global IT outage caused by a cybersecurity provider through carefully selected relevant articles.
To our customers and partners (Crowdstrike)
Technical details about how a content detection improvement caused the biggest global IT outage (Crowdstrike)
Channel File 291 controls how Falcon evaluates named pipe1 execution on Windows systems. Named pipes are used for normal, interprocess or intersystem communication in Windows.
The update that occurred at 04:09 UTC was designed to target newly observed, malicious named pipes being used by common C2 frameworks in cyberattacks. The configuration update triggered a logic error that resulted in an operating system crash.
Taviso with some thoughts about someone pointing out the issue was caused due to a NULL pointer. (Tavis Ormandy)
What I learned from the Microsoft Global IT Outage (Kevin Beaumont)
Technical details in 6 tweets and the reason Windows could not recover itself (Sergio de Los Santos)
Recent job advertisment for Crowdstrike (Linkedin - 22 July, 2024).
From GPT4 to AGI / from AGI to Superintelligence
En relacion al tema de la IA que muy brevemente expuse en la breve presentacion sobre IA y Ciberseguridad, aqui dejo un extenso documento escrito desde la vision de una de las notables figuras (y muy joven) en IA Leopold Aschenbrenner.
Todo el mundo, no importa cual sea tu interes en IA, deberia leer esto.
Press release of a Swiss based startup called FinalSpark.
Wetware computing, an exciting new frontier at the intersection of electrophysiology and artificial intelligence, uses living neurons to perform computations. Unlike artificial neural networks (ANNs), where digital weights can be updated instantly, biological neural networks (BNNs) require entirely new methods for network response modification. This complexity necessitates a system capable of conducting extensive experiments, ideally accessible to researchers globally.
The neuroplatform
A team at FinalSpark has developed a groundbreaking hardware and software system, the Neuroplatform, designed to enable electrophysiological experiments on a massive scale. The Neuroplatform allows researchers to conduct experiments on neural organoids, which can last over 100 days. This platform streamlines the experimental process, enabling quick production of new organoids, 24/7 monitoring of action potentials, and precise electrical stimulations. Additionally, an automated microfluidic system ensures stable environmental conditions by managing medium flow and changes without physical intervention.
Unprecedented Data Collection and Remote Access
Over the past three years, the Neuroplatform has been used to study over 1,000 brain organoids, generating more than 18 terabytes of data. A dedicated Application Programming Interface (API) supports remote research via Python libraries or interactive tools like Jupyter Notebooks. The API not only facilitates electrophysiological operations but also controls pumps, digital cameras, and UV lights for molecule uncaging. This setup allows for complex, continuous experiments incorporating the latest deep learning and reinforcement learning libraries.
Energy Efficiency and Future Prospects
The energy efficiency of wetware computing presents a compelling alternative to traditional ANNs. While training large language models (LLMs) like GPT-4 requires significant energy—up to 10 GWh per model—the human brain operates with approximately 86 billion neurons on just 20 W of power. This stark contrast underscores the potential of BNNs to revolutionize computing with their energy-efficient operation.
Scientific publication detailing FinalSpark’s Neuroplatform: “Open and remotely accessible Neuroplatform for research in wetware computing”
Exceptional piece of investigative journalism detailing the internal corporate fights to warn about a ticking bomb type of flaw "Golden SAML".
“Azure was the Wild West, just this constant race for features and functionality,”
“You will get a promotion because you released the next new shiny thing in Azure. You are not going to get a promotion because you fixed a bunch of security bugs.”
Product managers had little motivation to act fast, if at all, since compensation was tied to the release of new, revenue-generating products and features. That attitude was particularly pronounced in Azure product groups, former MSRC members said, because they were under pressure from Nadella to catch up to Amazon.
The ProPublica article reveals internal practices at Microsoft that prioritized new features over security for years, aiming to establish Azure as the leading cloud platform. This approach involved downplaying security issues, which enabled state actors to exploit these vulnerabilities. When Russian hackers breached SolarWinds' network management software, they did leverage post-exploit weaknesses, as the Golden SAML that Andrew was trying to warn about during years, to steal sensitive data and emails from the CLOUD.
Finally, these practices contributed to the Exchange compromise by Chinese actors, which eventually led to a highly critical report from the Cyber Safety Review Board.
Ref: https://www.propublica.org/article/microsoft-solarwinds-golden-saml-data-breach-russian-hackers
Update on 14/6/24 - both sites remain active.
According to my paper published in 2017, the median lifespan of a fraudulent website was one and a half years."
OpenAI, the company whose mission is: to build a safe and beneficial AGI, has released a report: AI and covert influence operations: latest trends
It seems it is the first of a series of report to show they combat the abuse of their platform. Few notes:
Attacker trends
Defender trends
Chronicle is now Google Security Operations. Say goodbye to legacy SIEMS
As cloud deployments for infrastructure, applications, and security have gained popularity, SecOps has had to evolve. Although many SIEM vendors claimed to offer cloud-native solutions, these were often superficial adjustments rather than genuine innovations addressing cloud security needs. At RSA 2019, Microsoft introduced "Azure Sentinel" (now Microsoft Sentinel) and Google introduced "Chronicle" (now Google Security Operations). Despite their progress, both have yet to fully address issues of coverage, effectiveness, and timeliness.
Europol's first report on the most threatening criminal networks active in the EU, unveils the presence of 821 dangerous criminal gangs across the EU, primarily engaged in drug trafficking and other illicit activities. These organizations operate transnationally, posing significant challenges to law enforcement. Efforts to combat them require enhanced coordination and initiatives to protect legal officials from intimidation and bribery.
There is a special section dedicated to Cyber-Attacks (pag 36) that highlights the disruption of lockbit ransomware group as case example:
The U.S. Department of Homeland Security (DHS) has issued the findings and recommendations report of the Cyber Safety Review Board (CSRB) concerning the review of the summer 2023 Microsoft Exchange Online Intrusion. This report is invaluable from all perspectives, particularly for professionals working in the CyberDetection field.
Following an initial review of the report, below are some of the key insights gleaned:
In today's hybrid environments, where organizations operate a mix of on-premises and cloud infrastructure, cybersecurity teams face daunting challenges in monitoring and securing their digital assets. While both Security Information and Event Management (SIEM) and Endpoint Detection and Response (EDR) solutions play crucial roles in defending against cyber threats, understanding their respective advantages is essential for maximizing cybersecurity effectiveness. This article explores the complexities of operating SIEM in hybrid environments, introduces the EDR topic, and highlights the advantages of SIEM over EDR in this context.
Complexities of operating SIEM in hybrid environments
EDR solutions focus on monitoring and securing endpoints, such as desktops, laptops, servers, and mobile devices, against advanced threats and malware. EDR platforms provide real-time visibility into endpoint activities, enabling rapid detection, investigation, and response to security incidents at the endpoint level. While EDR solutions excel in endpoint-focused threat detection and response, their scope is limited compared to the broader visibility offered by SIEM.
In the dynamic landscape of hybrid environments, achieving robust cybersecurity while managing costs is paramount for organizations. One strategy to balance these priorities involves leveraging SIEM solutions over EDR, particularly by harnessing the concept of security-relevant telemetry.
Security-relevant telemetry refers to the collection of essential security data, such as logs, network traffic, and endpoint activities, that are indicative of potential threats. By focusing on telemetry that directly contributes to threat detection and response, organizations can optimize their cybersecurity investments and avoid unnecessary data collection, usually tied to compliance related activities, that may inflate costs.
Security-relevant telemetry provides contextual insight into security events and incidents, enabling more accurate threat detection and response. By correlating telemetry data across diverse sources, including on-premises servers and cloud platforms, organizations can gain a holistic view of their hybrid environment's security posture without the need for additional security tools or solutions.
SIEM solutions offer centralized visibility into security-relevant telemetry across hybrid environments, enabling organizations to monitor and analyze essential security data in real-time. By aggregating and correlating telemetry data from diverse sources, SIEM enhances threat detection capabilities while minimizing costs associated with managing multiple security tools or platforms.
Prioritizing SIEM's security-relevant telemetry over EDR allows organizations to optimize their cybersecurity investments by focusing resources on data that directly contribute to threat detection and response. By eliminating unnecessary data collection and analysis, organizations can reduce operational costs associated with managing and maintaining security tools, ultimately maximizing cost efficiency in hybrid environments.
Organizations should conduct a thorough assessment of their security telemetry needs and rationalize data collection efforts to focus on security-relevant telemetry. This involves identifying critical security data sources and configuring SIEM solutions to prioritize telemetry that aligns with threat detection and response objectives.
To maintain cost efficiency over time, organizations should continuously optimize their telemetry collection and analysis processes based on evolving cybersecurity requirements and threat landscapes. This includes refining correlation rules, adjusting data retention policies, and leveraging automation to streamline telemetry management operations.
While both SIEM and EDR solutions are essential components of a robust cybersecurity strategy, organizations operating in hybrid environments can benefit significantly from leveraging the advantages offered by SIEM over EDR. By providing centralized visibility, comprehensive threat detection, and seamless integration with cloud environments, SIEM empowers organizations to effectively monitor, detect, and respond to security threats across diverse on-premises and cloud infrastructure. As organizations continue to navigate the complexities of hybrid environments, investing in robust SIEM solutions will be essential for maximizing cybersecurity effectiveness and safeguarding against evolving threats.