huginn

It's fair to point out that not all development is Internet connected, but ~58% of developers work in web dev.

5% in desktop apps

3% in mobile

2.4% in embedded

And then of the remaining I'd be shocked if few of their domains excluded Internet facing devices.

https://survey.stackoverflow.co/2023/#section-developer-roles-developer-type

But you're right to point out development isn't a monolith. Professionally though: anyone working in a field where cybersecurity is a concern should be thinking about and knowledgeable of cybersec.

I'm confused where cybersec sits in your sandwich analogy. If every time you sold a sandwich someone could use it to steal all the money in your business you'd probably need to know how to prevent reverse sandwich cashouts.

I'm not talking about advanced, domain specific cybersec. I don't expect every developer to have the sum total knowledge of crowd strike... But in a business environment I don't see how a developer can not consider cybersec in the code they write. Maybe in an org that is so compartmentalized down that you only own a single feature?

Idk I'm not sure I'd trust any dev who doesn't consider cyber security in their coding. So much development is centered around security whether that's auth or input sanitization or SQL query parameterization...

If you're working on an internal only application with no Internet connectivity then maybe you can ignore cybersec. But only maybe.

There's a wide variety of types of programming. It's nice that the core concepts can carry across between the disparate branches.

If I'm doing a particular custom view I'll end up using sin cos tan for some basic trig but that's about as complex as any mobile CRUD app gets.

I'm sure there are some math heavy mobile apps but they're the exception that proves the rule.

The scholastic discipline deserves that kind of nuance and Dijkstra was one of the greatest.

The practical discipline requires you build your work around specific computers. Much of the hard earned domain knowledge I've earned as a staff software engineer would be useless if I changed the specific computer it's built around - Android OS. An android phone has very specific APIs, code patterns and requirements. Being ARM even it's underlying architecture is fundamentally different from the majority of computers (for now. We'll see how much the M1 arm style arch becomes the standard for anyone other than Mac).

If you took a web dev with 10YOE and dropped them into my Android code base and said "ok, write" they should get the structure and basics but I would expect them to make mistakes common to a beginner in Android, just as if I was stuck in a web dev environment and told to write I would make mistakes common to a junior web dev.

It's all very well and good to learn the core of CS: the structures used and why they work. Classic algorithms and when they're appropriate. Big O and algorithmic complexity.

But work in the practical field will always require domain knowledge around specific computer features or even specific computers.

Working through a response on mobile so it's a bit chunked. I'll answer each point in series but it may take a bit.

1. that's not really what the video above claims. The presenter explicitly states that he believes GPT4 is intelligent, and that increasing the size of the LLM will make it true AGI. My entire premise here is not that an LLM is useless but that AGI is still entirely fantastical. Could an LLM conceivably be some building block of AGI? Sure, just like it could not be a building block of AGI. The "humble" position here is keeping AGI out of the picture because we have no idea what that is or how to get there, while we do know exactly what an LLM is and how it works. At its core an LLM is a complex dictionary. It is a queryable encoding of all the data that was passed through it.

Can that model be tweaked and tuned and updated? Sure. But there's no reason to think that it demonstrates any capability out of the ordinary for "queryable encoded data", and plenty of questions as to why natural language would be the queryable encoding of choice for an artificial intelligence. Your brain doesn't encode your thoughts in English, or whatever language your internal thoughts use if you're ESL+, language is a specific function of the brain. That's why damage to language centers in the brain can render people illiterate or mute without affecting any other capacities.

I firmly believe that LLMs as a component of broader AGI is certainly worth exploring just like any of the other hundreds of forms of genetic models or specialized "AI" tools: but that's not the language used to talk about it. The overwhelming majority of online discourse is AI maximalist, delusional claims about the impending singularity or endless claims of job loss and full replacement of customer support with ChatGPT.

Having professionally worked with GitHub Copilot for months now I can confidently say that it's useful for the tasks that any competent programmer can do as long as you babysit it. Beyond that any programmer who can do the more complex work that an LLC can't will need to understand the basics that an LLC generates in order to grasp the advanced. Generally it's faster for me to just write things myself than it is for Copilot to generate responses. The use cases I've found where it actually saves any time are:

1. Generating documentation (has at least 1 error in every javadoc comment that you have to fix but is mostly correct). Trying documentation first and code generated from it never worked well enough to be worth doing.

2. Filling out else cases or other branches of unit test code. Once you've written a pattern for one test it stamps out the permutations fairly well. Still usually has issues.

3. Inserting logging statements. I basically never have to tweak these, except prompting for more detail by writing a ,

This all is expected behavior for a model that has been trained on all examples of code patterns that have ever been uploaded online. It has general patterns and does a good job taking the input and adapting it to look like the training data.

But that's all it does. Fed more training data it does a better job of distinguishing patterns, but it doesn't change its core role or competencies: it takes an input and tries to make it's pattern match other examples of similar text.

I can't take that guy seriously. 16 minutes in he's saying the model is learning while also saying it's entirely frozen.

It's not learning, it's outputting different data that was always encoded in the model because of different inputs.

If you taught a human how to make a cake and they recited it back to you and then went and made a cake a human demonstrably learned how to make a cake.

If the LLM recited it back to you it's because it either contained enough context in its window to still have the entire recipe and then ran it through the equivalent of "summarize this - layers" OR it had the entire cake recipe encoded already.

No learning, no growth, no understanding.

The argument of reasoning is also absurd. LLMs have not been shown to have any emergent properties. Capabilities are linear progress based on parameters size. This is great in the sense that scaling model size means scaling functionality but it is also directly indicative that "reason" is nothing more than having sufficient coverage of concepts to create models.

Which of course LLMs have models: the entire point of an LLM is to be an encoding of language. Pattern matching the inputs to the correct model improves as model coverage improves: that's not unexpected, novel or even interesting.

What happens as an LLM grows in size is that decreasingly credulous humans are taken in by anthropomorphic bias and fooled by very elaborate statistics.

I want to point out that the entire talk there is self described as non-quantitative. Quantitative analysis of GPT4 shows it abjectly failing at comparatively simple abstract reasoning tests, one of the things he claims it does well. Getting a 33% on a test that the average human gets above 90% on is a damn bad showing, barely above random chance.

LLMs are not intelligent, they're complex.

But even in their greatest complexity they entirely fail to come within striking distance of even animal intelligence, much less human.

Do you comprehend how complex your mind is?

There are hundreds of neural transmitters in your brain. 20 billion neocortical neurons and an average 7 thousand connections per neuron. A naive complexity of 2.8e16 combinations. Each thought tweaking those ~7000 connections as it passes from neuron to neuron. The same thought can bounce between neurons, each time the signal getting to the same neuron it gets changed by the previous path, how long it has been since it last fired and the strengthened or weakened connection from other firings.

If you compare parameters complexity to neural complexity that puts the average, humdrum human mind at 20,000x the complexity of a model that cost billions to train and make... Which is also static. Only changed manually when they get into trouble or find bettI can't take that guy seriously. 16 minutes in he's saying the model is learning while also saying it's entirely frozen.

It's not learning, it's outputting different data that was always encoded in the model because of different inputs.

If you taught a human how to make a cake and they recited it back to you and then went and made a cake a human demonstrably learned how to make a cake.

If the LLM recited it back to you it's because it either contained enough context in its window to still have the entire recipe and then ran it through the equivalent of "summarize this - layers" OR it had the entire cake recipe encoded already.

No learning, no growth, no understanding.

The argument of reasoning is also absurd. LLMs have not been shown to have any emergent properties. Capabilities are linear progress based on parameters size. This is great in the sense that scaling model size means scaling functionality but it is also directly indicative that "reason" is nothing more than having sufficient coverage of concepts to create models.

Which of course LLMs have models: the entire point of an LLM is to be an encoding of language. Pattern matching the inputs to the correct model improves as model coverage improves: that's not unexpected, novel or even interesting.

What happens as an LLM grows in size is that decreasingly credulous humans are taken in by anthropomorphic bias and fooled by very elaborate statistics.

I want to point out that the entire talk there is self described as non-quantitative. Quantitative analysis of GPT4 shows it abjectly failing at comparatively simple abstract reasoning tests, one of the things he claims it does well. Getting a 33% on a test that the average human gets above 90% on is a damn bad showing, barely above random chance.

LLMs are not intelligent, they're complex.

But even in their greatest complexity they entirely fail to come within striking distance of even animal intelligence, much less human.

Do you comprehend how complex your mind is?

There are hundreds of neural transmitters in your brain. 20 billion neocortical neurons and an average 7 thousand connections per neuron. A naive complexity of 2.8e16 combinations. Each thought tweaking those ~7000 connections as it passes from neuron to neuron. The same thought can bounce between neurons, each time the signal getting to the same neuron it gets changed by the previous path, how long it has been since it last fired and the strengthened or weakened connection from other firings.

If you compare parameters complexity to neural complexity that puts the average, humdrum human mind at 20,000x the complexity of a model that cost billions to train and make... Which is also static. Only changed manually when they get into trouble or find better optimizations.

And it's still deeply flawed and incapable of most tasks. It's just very good at convincing you with generalizations.

Since the pandemic antivax is overwhelmingly conservative.

Before then it was fringe hippies... who were also often conservative, they just smoked weed as well.

The UK has an atrocious train system in terms of cost.

Point fingers where they belong: your own government.

I'm voting in absentia for my 400 frozen embryos. They're all 18.

Given that when the servers are overloaded you don't get rewards: They probably have a centralized database or some other centralization issue.

You can add to a cluster all you want but if you're using a SQL monolith you'll get locks.

If you only have 100k concurrent users you can probably get away with it.

At 450k they're running into gridlock