Hisham ElRazaz

Proud to share my latest article about LLM data preparation

11

#buildinginpublic #pccloner Post 18/N After 20+ hours of work distributed over two and a half weeks, I finally built the first complete version of this architecture from DeepMind. Of course, I had about 12 points of where I'm not sure what the authors did, but I know exactly where those points are and I can make changes easily in most of them. I still have to train this model, but up to today it is working well with expected data from miniwob++ benchmark. This model has more moving parts than I thought. Paper: https://lnkd.in/di6hH29m

1

𝗛𝗼𝘁 𝘁𝗮𝗸𝗲: 𝗮𝗿𝗲 𝗟𝗮𝗿𝗴𝗲 𝗟𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗠𝗼𝗱𝗲𝗹𝘀 𝙖𝙘𝙩𝙪𝙖𝙡𝙡𝙮 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝘁? Many fellow AI experts I talk to argue that the revolutionary capabilities of Large Language Models (LLMs) such as ChatGPT are just echos of their training data, that they are “just fancy statistics”, or “just next-word predictors”, suggesting that true Artificial General Intelligence (AGI) will require more sophisticated technological advancements yet to be discovered. I disagree. Yes, the basic building blocks of neural networks are hardly sophisticated, much like a single cogwheel in a large machine. But the principle of complexity emerging from simple rules is well-documented and, more importantly, ubiquitous. Take the Game of Life, devised by John Conway back in 1970, which you play by creating an initial configuration on a grid of cells and observing how it evolves. The rules of the Game of Life are extremely simple: 1. 𝗕𝗶𝗿𝘁𝗵: A cell that is dead at time (t) will be alive at time (t+1) if 3 of its 8 neighbors were alive at time (t). 2. 𝗦𝘂𝗿𝘃𝗶𝘃𝗮𝗹: A cell stays alive at time (t+1) if 2 or 3 of its neighbors are alive at time (t); otherwise, it dies or remains dead. 3. 𝗗𝗲𝗮𝘁𝗵: A cell dies from isolation if it has 1 or fewer alive neighbors, and it dies from overpopulation if it has 4 or more alive neighbors. These 3 simple rules can produce a vast range of outcomes, including stable structures, oscillators (which cycle back to their initial state), and even structures that seem to "travel" across the board. All these different, complex behaviors emerge from just this set of simple rules, interacting on a grid. Similarly, fractals, with their endless complexity, arise from very straightforward mathematical equations. These patterns, found both in nature and mathematics, illustrate how simple, repetitive processes can generate infinitely complex structures and behaviors. You may think these examples are a bit abstract or contrived, but nature itself is full of complexity arising from simplicity. Take swarm intelligence, where individuals (like ants, birds or bacteria) perform simple actions based only on 𝙡𝙤𝙘𝙖𝙡 information. Despite the simplicity of their actions, the collective exhibits remarkable intelligence, solving complex problems and adapting to 𝙜𝙡𝙤𝙗𝙖𝙡𝙡𝙮 changing environments with remarkable efficiency. And let’s address the elephant in the room: isn’t the human brain, the epitome of complexity, not also “just” a collection of brain cells? Billions of neurons, each following basic biological rules, interact to produce the rich mosaic of human thought, emotion, and consciousness. Complexity from simplicity isn’t just theory; it’s a pattern that underpins our universe. As LLMs evolve, the convergence of simple computational processes may very well pave the way for AGI, challenging our notions of what machines can achieve. What do you think? #AI #LLM #AGI #GenAI

24

2 Kommentare