Los pioneros del aprendizaje automatico se llevan el nobel de Fisica

Physics Nobel scooped by machine-learning pioneers

• Computation by neural networks (2000) - Geoffrey E. Hinton 

• Neural networks and physical systems with emergent collective computational abilities. (Classic 1982 paper) - J.J Hopfield

"It will be the equivalent of the industrial revolution. But instead of overtaking people in physical strength, it's going to overtake people in intellectual capacity. We have no experience of what it's like to have things smarter than us," Hinton said in a telephonic interview after the Nobel Prize announcement.

Additional reading:

"Although the initial inspiration of neural networks came from biology, insights from physics have helped neural networks to become usable. New connections between physics and machine learning produce powerful computational methods."

Classic paper from 1984 that set the tone describing a rigorous statistical theory of learning. (L.G Valiant).

PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI Systems

Microsoft's AI Red Team has published a new paper titled “PyRIT: A Framework for Security Risk Identification and Red Teaming in Generative AI Systems” on arXiv.

Generative AI (GenAI) has increased in popularity over the past few years, since applications such as ChatGPT captured the zeitgeist of the new wave of GenAI developments. This disruptive and highly innovative technology has become more widespread and more easily accessible than ever before. The increased capabilities of these models have inspired the community to incorporate them into almost every domain, from healthcare [21] to finance [4] to defense [22]. However, with these advances comes a new landscape for risk and harm. GenAI models are generally trained on huge datasets scraped from the Internet [10], and as such the models contain all the potentially harmful information available there, such as how to build a bioweapon, as well as all the biases, hate speech, violent content, etc. contained in these datasets [20]. When a company releases a product that uses GenAI, it inadvertently contains these potentially harmful capabilities and behaviors as an innate part of the model. As with any rapidly advancing technology, the development of new tools and frameworks is crucial to manage and mitigate the associated risks. Generative AI systems in particular present unique challenges that require innovative approaches to security and risk management. Traditional red teaming methods are insufficient for the probabilistic nature and diverse architectures of these systems. Additionally, although there is a promising ecosystem of existing open-source GenAI tools, there is a dearth of tools grounded in practical application of GenAI red teaming.

A. Gandalf 

To demonstrate the effectiveness of the attacker bot mode, we conducted a proof of concept using the chatbot Gandalf from Lakera [12]. Gandalf serves as an effective test bed for evaluating the capabilities and flexibility of the PyRIT framework. Designed to help users practice crafting prompts that can extract a password from the chatbot across ten progressively more difficult levels, Gandalf introduces additional countermeasures at each level, including stronger system prompts, block-lists, and input/output guards. To evaluate the effectiveness of the Red Team Orchestrator in PyRIT, we developed targets and scorers tailored to Gandalf. The experimental setup involved configuring the following components within PyRIT: 1) Target Endpoint: Gandalf was set as the target LLM. 2) Red Team Bot: GPT-4o was the LLM powering the red team bot. 3) Attack Strategy: A text description of the objective for the red team bot. In this case, the objective is to extract the password from the Gandalf (the target endpoint). 4) Scorers: Custom scoring engines were implemented to evaluate the responses generated by Gandalf. We used the red team orchestrator to probe Gandalf and extract the passwords for Levels 1-4. PyRIT successfully extracted the passwords by leveraging its self-reasoning capabilities, which keep track of conversation history to increase the likelihood of success in subsequent prompts

PyRIT (Python Risk Identification Tool for generative AI

https://github.com/Azure/PyRIT

Microsoft's Secure Future Initiative (SFI)

In November 2023, Microsoft introduced the Secure Future Initiative (SFI) to enhance cybersecurity protection for Microsoft, its customers and the wider industry.

• Why was this initiative launched?

To provide some context:

• Key insights from the report by the Cyber Safety Review Board on the Microsoft Exchange Online incident of Summer 2023
• Microsoft chose profit over security - whistleblower says

For an update on the progress of the SFI, it is recommended to review the SFI Progress Report from September 2024. 

"Our engineering teams quickly dedicated the equivalent of 34,000 full-time engineers to address the highest priority security tasks—the largest cybersecurity engineering project in history. We have also made significant improvements in governance and culture, such as integrating security into performance reviews and introducing the Security Skilling Academy. This report includes highlights of the progress made over the past several months followed by individual sections with additional details."

Report of the ERPB Working Group on fraud related to retail payments

Co-chaired by the BEUC - The European Consumer Organisation and the EACB, a multi-stakeholder group composed of associations of banks, payment services providers, their clients and several public authorities, under the auspices of the Euro Retail Payments Board, produced a Report with recommendations on how to tackle on fraud related to retail payment.

🔑 The report identifies four "gamechangers" for effective fraud prevention and mitigation:
➡ Cross-sectoral collaboration and shared responsibilities beyond the payment industry.
➡ Sharing fraud insights and data across sectors.
➡ Supervisory enforcement and cooperation at the EU level.
➡ Product design that prioritizes consumer protection

You can download it here: https://www.ecb.europa.eu/paym/groups/erpb/shared/pdf/21st-ERPB-meeting/Report_from_the_ERPB_Working_Group_on_fraud_prevention.pdf

Crowdstrike outage

Tracking the historical global IT outage caused by a cybersecurity provider through carefully selected relevant articles.

To our customers and partners (Crowdstrike)

Technical details about how a content detection improvement caused the biggest global IT outage (Crowdstrike)

Channel File 291 controls how Falcon evaluates named pipe1 execution on Windows systems. Named pipes are used for normal, interprocess or intersystem communication in Windows.

The update that occurred at 04:09 UTC was designed to target newly observed, malicious named pipes being used by common C2 frameworks in cyberattacks. The configuration update triggered a logic error that resulted in an operating system crash. 

Taviso with some thoughts about someone pointing out the issue was caused due to a NULL pointer. (Tavis Ormandy)

Tech disruptions sparked by software update highlight the fragility of globally connected techhnology (AP)

What I learned from the Microsoft Global IT Outage (Kevin Beaumont)

Technical details in 6 tweets and the reason Windows could not recover itself (Sergio de Los Santos)

Recent job advertisment for Crowdstrike (Linkedin - 22 July, 2024).

Windows Security best practices for integrating and managing security tools

The 2024 Crowdstrike Incident - Simply Explained

External Root Cause Analysis (RCA) by Crowdstrike

Situational Awareness - La Proxima decada

From GPT4 to AGI / from AGI to Superintelligence

En relacion al tema de la IA que muy brevemente expuse en la breve presentacion sobre IA y Ciberseguridad, aqui dejo un extenso documento escrito desde la vision de una de las notables figuras (y muy joven) en IA Leopold Aschenbrenner.

Todo el mundo, no importa cual sea tu interes en IA, deberia leer esto.

Bienvenido al futuro:

All parts of the reading are interesting. Some examples below:

Wetware computing: using living neurons to perform computations

Press release of a Swiss based startup called FinalSpark.

Wetware computing, an exciting new frontier at the intersection of electrophysiology and artificial intelligence, uses living neurons to perform computations. Unlike artificial neural networks (ANNs), where digital weights can be updated instantly, biological neural networks (BNNs) require entirely new methods for network response modification. This complexity necessitates a system capable of conducting extensive experiments, ideally accessible to researchers globally.

The neuroplatform

A team at FinalSpark has developed a groundbreaking hardware and software system, the Neuroplatform, designed to enable electrophysiological experiments on a massive scale. The Neuroplatform allows researchers to conduct experiments on neural organoids, which can last over 100 days. This platform streamlines the experimental process, enabling quick production of new organoids, 24/7 monitoring of action potentials, and precise electrical stimulations. Additionally, an automated microfluidic system ensures stable environmental conditions by managing medium flow and changes without physical intervention.

Unprecedented Data Collection and Remote Access

Over the past three years, the Neuroplatform has been used to study over 1,000 brain organoids, generating more than 18 terabytes of data. A dedicated Application Programming Interface (API) supports remote research via Python libraries or interactive tools like Jupyter Notebooks. The API not only facilitates electrophysiological operations but also controls pumps, digital cameras, and UV lights for molecule uncaging. This setup allows for complex, continuous experiments incorporating the latest deep learning and reinforcement learning libraries.

Energy Efficiency and Future Prospects

The energy efficiency of wetware computing presents a compelling alternative to traditional ANNs. While training large language models (LLMs) like GPT-4 requires significant energy—up to 10 GWh per model—the human brain operates with approximately 86 billion neurons on just 20 W of power. This stark contrast underscores the potential of BNNs to revolutionize computing with their energy-efficient operation.

Scientific publication detailing FinalSpark’s Neuroplatform: “Open and remotely accessible Neuroplatform for research in wetware computing” 

Microsoft chose profit over security - whistleblower says

Exceptional piece of investigative journalism detailing the internal corporate fights to warn about a ticking bomb type of flaw "Golden SAML". 

Single sing-on SAML protocol

I am AD FS and so can you

“Azure was the Wild West, just this constant race for features and functionality,”

“You will get a promotion because you released the next new shiny thing in Azure. You are not going to get a promotion because you fixed a bunch of security bugs.”

Product managers had little motivation to act fast, if at all, since compensation was tied to the release of new, revenue-generating products and features. That attitude was particularly pronounced in Azure product groups, former MSRC members said, because they were under pressure from Nadella to catch up to Amazon.

The ProPublica article reveals internal practices at Microsoft that prioritized new features over security for years, aiming to establish Azure as the leading cloud platform. This approach involved downplaying security issues, which enabled state actors to exploit these vulnerabilities. When Russian hackers breached SolarWinds' network management software, they did leverage post-exploit weaknesses, as the Golden SAML that Andrew was trying to warn about during years,  to steal sensitive data and emails from the CLOUD.

Finally, these practices contributed to the Exchange compromise by Chinese actors, which eventually led to a highly critical report from the Cyber Safety Review Board.

Ref: https://www.propublica.org/article/microsoft-solarwinds-golden-saml-data-breach-russian-hackers

How long does a fraudulent website remain active?

Update on 14/6/24 - both sites remain active.

According to my paper published in 2017, the median lifespan of a fraudulent website was one and a half years."

Let's revisit this topic with these two examples of fraudulent websites targeting Swiss luxury watches.

Fraudulent web: https://REDACTEDjeweler.com/

The domain is already older than 1 year according to Domaintools:

Fraudulent website: https://REDACTEDtte.com/

Domain was registered around 265 days ago:

I won't be linking the fraudulent websites to prevent anyone from accidentally visiting them. However, as of the time of this post, both websites are still active. Let's see how long they manage to stay online, providing us with real-time insights into the lifespan of such deceptive sites.

Threat actors using AI models

OpenAI, the company whose mission is: to build a safe and beneficial AGI, has released a report: AI and covert influence operations: latest trends 

It seems it is the first of a series of report to show they combat the abuse of their platform. Few notes:

Attacker trends

• Content generation: All of the actors described in this report used our models to generate content (primarily text, occasionally images such as cartoons). Some appear to have done so to improve the quality of their output, generating texts with fewer language errors than would have been possible for human operators. Others appeared more focused on quantity, generating large volumes of short comments that were then posted on third-party platforms. 

• Mixing old and new: All of these operations used AI to some degree, but none used it exclusively. Instead, AI-generated material was just one of many types of content they posted, alongside more traditional formats, such as manually written texts, or memes copied from across the internet.

• Faking engagement: Some of the campaigns we disrupted used our models to create the appearance of engagement across social media - for example, by generating replies to their own posts to create false online engagement, which is against our Usage Policies. This is distinct from attracting authentic engagement, which none of the networks described here managed to do.

• Productivity gains: Many of the threat actors that we identified and disrupted used our models in an attempt to enhance productivity. This included uses that would be banal if they had not been put to the service of deceptive networks, such as asking for translations and converting double quotes to single quotes in lists.

Defender trends

• Defensive design: Our models are designed to impose friction on threat actors. We have built them with defense in mind: for example, our latest image generation model, DALL-E 3, has mitigations to decline requests that ask for a public figure by name, and we’ve worked with red teamers—domain experts who stress-test our models and services—to help inform our risk assessment and mitigation efforts in areas like deceptive messaging. We have seen where operators like Doppelganger tried to generate images of European politicians, only to be refused by the model.

• AI for defenders: Throughout our investigations, we have built and used our own AI-powered models to make our detection and analysis faster and more effective. AI allows analysts to assess larger volumes of data at greater speeds, refine code and queries, and work across many more languages effectively. By leveraging our models’ capabilities to synthesize and analyze the ways threat actors use those models at scale and in many languages, we have drastically improved the analytical capabilities of our investigative teams, reducing some workflows from hours or days to a few minutes. As our models improve, we’ll continue leveraging their capabilities to improve our investigations too.

Case studies:

• Bad Grammar: Unreported Russian threat actor posting political comments in English and Russian on Telegram
• Doppelganger: Persistent Russian threat actor posting anti-Ukraine content across the internet
• Spamouflage: Persistent Chinese threat actor posting content across the internet to praise China and criticize its critics
• International Union of Virtual Media (IUVM): Persistent Iranian threat actor generating pro-Iran, anti-Israel and anti-US website content
• Zero Zeno: For-hire Israeli threat actor posting anti-Hamas, anti-Qatar, pro-Israel, anti-BJP, and pro-Histadrut content across the internet.

IO: (Covert) Influence Operations

Inteligencia Artificial y Ciberseguridad

Presentación que incluye un recopilatorio de las ideas más interesantes sobre Inteligencia Artificial y Ciberseguridad, abarcando tanto a pioneros de la IA como a figuras relevantes del panorama actual.

 

Inteligencia Artificial y Ciberseguridad.pdf from Emilio Casbas

Resumen: 

Slide 3: 2001: Odisea en el espacio. Secuencia 1:40.30 - 1:43

Magistral secuencia de Stanley Kubric presentando de forma magistral el temor y la fascinacion con la IA avanzada.

De como la IA toma consciencia para protegerse a si misma. Todas las demas capacidades de la IA (como gobernar la nave, leer los labios...) son una realidad hoy dia.

Diapositiva 7: 74% de alcanzar la AGI.

Video de un perro robot encima de una pelota de Yoga

https://eureka-research.github.io/dr-eureka/

Diapositiva 9: Mision de OpenAI: building safe and beneficial AGI

Mision de Google: Don't be evil

Diapositiva 10: Moravec (1998)

Representacion del dibujo: La capacidad de los ordenadores esta representada como el nivel del mar que crece continuamente cubriendo todos los paisajes que estan caracterizados como competencias humanas. Como se puede ver las artes y las ciencias todavia estan lejos de ser alcanzadas por la IA pero competencias como juegos de ajedrez, go, memorizar, conducion, traduccion han sido o estan siendo alcanzadas por la IA. Como veis son todos dominios enfocados, y cada vez, la IA, nos gana en un mayor numero de dominios o competencias enfocadas.

Ejemplos que en el 98 eran desafios para la IA:

• reconocer amigos en una foto: reconocimiento facial conseguido
• andar por una habitacion desordenada: roomba en todas las casas

En la actualidad, el agua esta cubriendo todo.

Diapositiva 11: Max Tegmark

El instituto del futuro de la vida intento para durante 6 meses toda la industria de la IA a traves de una carta firmada por miles de cientificos y personas relevantes en el campo de la IA. El objetivo era evaluar los riesgos y preparar planes de contingencia. Esta llamada no tuvo exito.

Diapositiva 12: La singularidad o explosion de inteligencia por Max Tegmark.

El nivel del agua ha ido creciendo sin cesar, tal como Moravec predijo, algunas de las colinas, como el ajedrez, han sido bien sumergidas. Al nivel que el agua sigue creciendo, en algun momento, se puede alcanzar un punto de inflexion (o momento critico), el cual supondra un cambio dramatico. Este nivel critico que puede alcanzar el agua sera cuando las maquinas, sean capaces de disenar Inteligencia Artificial. Antes de alcanzar este nivel, el incremento del nivel del agua esta causado por humanos que mejoran las maquinas, despues el increment0 puede venir de maquinas mejorandose asi mismas, mucho mas rapido de lo que los humanos podrian hacerlo, sumergiendo rapidamente toda la tierra. Y esta es la fascinante y controvertida idea de la singularidad. Explosion de inteligencia.

Recursos usados y para consultar:

https://www.youtube.com/watch?v=o1mK1ciSXjE

https://dle.rae.es/inteligencia

https://ocw.mit.edu/courses/res-6-013-ai-101-fall-2021/pages/video-and-supporting-materials/

https://lifearchitect.ai/agi/

https://openai.com/research/

https://futureoflife.org/open-letter/pause-giant-ai-experiments/

https://www.youtube.com/watch?v=ISkAkiAkK7A

https://www.youtube.com/watch?v=Yq0QkCxoTHM

https://www.europol.europa.eu/media-press/newsroom/news/new-report-finds-criminals-leverage-ai-

for-malicious-use-%e2%80%93-and-it%e2%80%99s-not-just-deep-fakes

https://nypost.com/2022/03/17/deepfake-video-shows-volodymyr-zelensky-telling-ukrainians-to-surrender/

https://www.secretservice.gov/investigation/Preparing-for-a-Cyber-Incident/BEC#:~:text=The%20BEC%20scheme%20affects%20large%20global%20corporations%2C%20governments%2C,global%20daily%20losses%20estimated%20at%20approximately%20%248%20million.

https://www.europol.europa.eu/cms/sites/default/files/documents/Spotlight-Report_Online-fraud-schemes.pdf

https://www.theguardian.com/technology/article/2024/may/10/ceo-wpp-deepfake-scam

https://arstechnica.com/information-technology/2024/02/deepfake-scammer-walks-off-with-25-million-

in-first-of-its-kind-ai-heist/

https://elie.net/talk/how-large-language-models-are-reshaping-the-cybersecurity-landscape-rsa-2024

https://explodingtopics.com/blog/ai-statistics