Situational Awareness - La Proxima decada

From GPT4 to AGI / from AGI to Superintelligence

En relacion al tema de la IA que muy brevemente expuse en la breve presentacion sobre IA y Ciberseguridad, aqui dejo un extenso documento escrito desde la vision de una de las notables figuras (y muy joven) en IA Leopold Aschenbrenner.

Todo el mundo, no importa cual sea tu interes en IA, deberia leer esto.

Bienvenido al futuro:

All parts of the reading are interesting. Some examples below:

Wetware computing: using living neurons to perform computations

Press release of a Swiss based startup called FinalSpark.

Wetware computing, an exciting new frontier at the intersection of electrophysiology and artificial intelligence, uses living neurons to perform computations. Unlike artificial neural networks (ANNs), where digital weights can be updated instantly, biological neural networks (BNNs) require entirely new methods for network response modification. This complexity necessitates a system capable of conducting extensive experiments, ideally accessible to researchers globally.

The neuroplatform

A team at FinalSpark has developed a groundbreaking hardware and software system, the Neuroplatform, designed to enable electrophysiological experiments on a massive scale. The Neuroplatform allows researchers to conduct experiments on neural organoids, which can last over 100 days. This platform streamlines the experimental process, enabling quick production of new organoids, 24/7 monitoring of action potentials, and precise electrical stimulations. Additionally, an automated microfluidic system ensures stable environmental conditions by managing medium flow and changes without physical intervention.

Unprecedented Data Collection and Remote Access

Over the past three years, the Neuroplatform has been used to study over 1,000 brain organoids, generating more than 18 terabytes of data. A dedicated Application Programming Interface (API) supports remote research via Python libraries or interactive tools like Jupyter Notebooks. The API not only facilitates electrophysiological operations but also controls pumps, digital cameras, and UV lights for molecule uncaging. This setup allows for complex, continuous experiments incorporating the latest deep learning and reinforcement learning libraries.

Energy Efficiency and Future Prospects

The energy efficiency of wetware computing presents a compelling alternative to traditional ANNs. While training large language models (LLMs) like GPT-4 requires significant energy—up to 10 GWh per model—the human brain operates with approximately 86 billion neurons on just 20 W of power. This stark contrast underscores the potential of BNNs to revolutionize computing with their energy-efficient operation.

Scientific publication detailing FinalSpark’s Neuroplatform: “Open and remotely accessible Neuroplatform for research in wetware computing” 

Microsoft chose profit over security - whistleblower says

Exceptional piece of investigative journalism detailing the internal corporate fights to warn about a ticking bomb type of flaw "Golden SAML". 

Single sing-on SAML protocol

I am AD FS and so can you

“Azure was the Wild West, just this constant race for features and functionality,”

“You will get a promotion because you released the next new shiny thing in Azure. You are not going to get a promotion because you fixed a bunch of security bugs.”

Product managers had little motivation to act fast, if at all, since compensation was tied to the release of new, revenue-generating products and features. That attitude was particularly pronounced in Azure product groups, former MSRC members said, because they were under pressure from Nadella to catch up to Amazon.

The ProPublica article reveals internal practices at Microsoft that prioritized new features over security for years, aiming to establish Azure as the leading cloud platform. This approach involved downplaying security issues, which enabled state actors to exploit these vulnerabilities. When Russian hackers breached SolarWinds' network management software, they did leverage post-exploit weaknesses, as the Golden SAML that Andrew was trying to warn about during years,  to steal sensitive data and emails from the CLOUD.

Finally, these practices contributed to the Exchange compromise by Chinese actors, which eventually led to a highly critical report from the Cyber Safety Review Board.

Ref: https://www.propublica.org/article/microsoft-solarwinds-golden-saml-data-breach-russian-hackers

How long does a fraudulent website remain active?

Update on 14/6/24 - both sites remain active.

According to my paper published in 2017, the median lifespan of a fraudulent website was one and a half years."

Let's revisit this topic with these two examples of fraudulent websites targeting Swiss luxury watches.

Fraudulent web: https://REDACTEDjeweler.com/

The domain is already older than 1 year according to Domaintools:

Fraudulent website: https://REDACTEDtte.com/

Domain was registered around 265 days ago:

I won't be linking the fraudulent websites to prevent anyone from accidentally visiting them. However, as of the time of this post, both websites are still active. Let's see how long they manage to stay online, providing us with real-time insights into the lifespan of such deceptive sites.

Threat actors using AI models

OpenAI, the company whose mission is: to build a safe and beneficial AGI, has released a report: AI and covert influence operations: latest trends 

It seems it is the first of a series of report to show they combat the abuse of their platform. Few notes:

Attacker trends

• Content generation: All of the actors described in this report used our models to generate content (primarily text, occasionally images such as cartoons). Some appear to have done so to improve the quality of their output, generating texts with fewer language errors than would have been possible for human operators. Others appeared more focused on quantity, generating large volumes of short comments that were then posted on third-party platforms. 

• Mixing old and new: All of these operations used AI to some degree, but none used it exclusively. Instead, AI-generated material was just one of many types of content they posted, alongside more traditional formats, such as manually written texts, or memes copied from across the internet.

• Faking engagement: Some of the campaigns we disrupted used our models to create the appearance of engagement across social media - for example, by generating replies to their own posts to create false online engagement, which is against our Usage Policies. This is distinct from attracting authentic engagement, which none of the networks described here managed to do.

• Productivity gains: Many of the threat actors that we identified and disrupted used our models in an attempt to enhance productivity. This included uses that would be banal if they had not been put to the service of deceptive networks, such as asking for translations and converting double quotes to single quotes in lists.

Defender trends

• Defensive design: Our models are designed to impose friction on threat actors. We have built them with defense in mind: for example, our latest image generation model, DALL-E 3, has mitigations to decline requests that ask for a public figure by name, and we’ve worked with red teamers—domain experts who stress-test our models and services—to help inform our risk assessment and mitigation efforts in areas like deceptive messaging. We have seen where operators like Doppelganger tried to generate images of European politicians, only to be refused by the model.

• AI for defenders: Throughout our investigations, we have built and used our own AI-powered models to make our detection and analysis faster and more effective. AI allows analysts to assess larger volumes of data at greater speeds, refine code and queries, and work across many more languages effectively. By leveraging our models’ capabilities to synthesize and analyze the ways threat actors use those models at scale and in many languages, we have drastically improved the analytical capabilities of our investigative teams, reducing some workflows from hours or days to a few minutes. As our models improve, we’ll continue leveraging their capabilities to improve our investigations too.

Case studies:

• Bad Grammar: Unreported Russian threat actor posting political comments in English and Russian on Telegram
• Doppelganger: Persistent Russian threat actor posting anti-Ukraine content across the internet
• Spamouflage: Persistent Chinese threat actor posting content across the internet to praise China and criticize its critics
• International Union of Virtual Media (IUVM): Persistent Iranian threat actor generating pro-Iran, anti-Israel and anti-US website content
• Zero Zeno: For-hire Israeli threat actor posting anti-Hamas, anti-Qatar, pro-Israel, anti-BJP, and pro-Histadrut content across the internet.

IO: (Covert) Influence Operations