When understanding a new "magic", there's this beautiful moment when you grok it, and the abstraction poofs away.
It's when you take apart a mechanical clock and keep looking for the time-keeping part, until you figure out that there isn't a time-keeping part in there, it's just gears and a spring.
It's when you learn about integrated circuits and full-adders, and keep trying to understand how a bunch of transistors can do Mathematics, until you figure out that there isn't a mathematics-doing part in there, it's just circuits and wires, arranged in a way that makes the voltages come out right.
It's when your understanding of the top-down structure snaps together with the bottom-up mechanics of the building blocks. There's no space left for the ghost in the machine to haunt, and you go "Oh. huh".
I live for that moment.
I went through an EE degree instead of a CS degree in undergrad specifically so I could peel back this magic and really understand what's going on, to some detail, down to the electromagnetism level. It is indeed a very freeing feeling to be resting atop so many layers of abstractions, with the understanding that if you ever had to, you could go down to any one of them and kind of feel your way around.
I think for me the biggest magic-killing moment was when I realized that CPU clock cycles, timing trees, etc. were all just ways for us to shoehorn a fundamentally analog thing (how much voltage is in area X?) to a digital thing (does area X have enough voltage? yes? OK, call it a 1 and let's move on already!). Somehow to me that feels like the ultimate "leaky" abstraction, although of course decades and trillions of dollars have gone into making it ever more watertight for 99.99-several-more-9s% of the time. At least to the end user. Your mileage may vary if you're a TSMC researcher or something, of course.
I've always felt the whole binary/digital thing was one of the most clever bits of compromise with the real world I've ever seen.
You have this thing, you want to be able to translate its value into something useful. In this case, the amount of voltage in a circuit to a number. And you spend so much time trying to make sure the voltage level passed is rock solid, that your read is equally solid, etc. Until you realize that you'd have to invent so many more industries just to do this one thing that you just give up and say the only thing you can know with certainty is that there is or is not voltage passing through the circuit.
Then you need to be able to translate "ON" and "OFF" into actual usable values. And eventually coming down to a base 2 counting system so that 4 circuits gives you 16 distinct values seems obvious in hindsight, but had to be a revelation when they realized it.
Bingo, precisely. I keep using this compromise as a foundational example when I talk to my wife (currently doing a CS degree) about why binary stuff, specifically, keeps coming up in her stuff. 2 is a magic number in computing because it is the Great Compromise we made with reality to get it.
Every now and then people here bring up more exotic paradigms, like ternary computing, and yes! In an ideal world the more phases you can detect the 'better', all the way up to infinity (pure analogue computing). But the difficulty curve to scale anything besides base 2 to where base 2 computing currently is is, at least for our current understanding in physics, materials science, etc., way higher.
Actually, Leibniz came up with the dual/binary system as being ideal for computation 300 years ago roughly around the same time he built his mechanical calculator, long before there was any chance of there being programmable computers.
I work with FPGAs and embedded systems occasionally, and you have no idea how amazingly watertight the phone/desktop/server CPU abstraction is in comparison to what you get the moment you do something slightly weird. A combination of the chips, the firmware, and the OS does so much work to give you the abstraction of "machine runs code and it just works."
There's a paper from the 1990s of someone trying to do neural net-type things to construct an FPGA, and it gets really out there, like 'this gate isn't actually connected to anything but if we change anything about it the laws of electromagnetism cause the whole thing to stop working' weird. FPGAs are fascinating little dudes, esp in domains like HFT where their speed directly translates into dollars, but I totally believe you here.
for me, it was latches and creating a CPU which instructions could be fed into. but it wasn't a "lost the magic" feeling, it was a "that's amazing!" feeling.
While the Foundation series is what's best known nowadays, the Suzan Calvin and Elijah Bailay stuff, and more generally everything set before the Galactic Empire even exists in Asimov's fictional timeline is where Asimov is at his best.
That's the difference between technology magic and illusionist magic, When you see how the trick is done with illusions it's always a bit of a letdown because the answer is usually prosaic, the 'magic' vanishes and it becomes a trick.
When you understand how a piece of technology works you get that beautiful moment.
I've never got that. I feel the same way in either case, if your trick was easy everybody would do it. Sleight of hand tricks for example, if you're good they're completely seamless, I could never hope to reproduce and yet I know exactly how it's done.
Take that Penn & Teller trick where the live audience is just lying - that's a bit lazy, we're not supposed to have some great admiration for this trick they're just showing you it's an option to just have the live audience lie to the recorded audience and that "works". Whereas their transparent version of cups and balls is the opposite, you must be a sleight of hand master or this won't look like anything.
My wife and I are part time magicians. In my experience, people who share your mindset become magicians.
At the risk of just repeating exactly what the person you're replying to wrote: The reason that non-magicians often feel let down when they find out how a trick is executed, is that it often feels like an insult to their intelligence. Magic toys with and exploits your assumptions. As soon as you find out that those assumptions were incorrect, and that you were just lied to by a prop (for example), your experience and perception of that illusion goes from "OMG" to "oh, that's all?"
What we want the methods to be is some grandiose Ocean's 11 like "heist" with tons of sneaky maneuvers and difficult sleight of hand ... but MOST magic tricks, including the ones that Penn & Teller fool you with (not the ones where they tip the method) are not that.
Penn & Teller have said themselves that (paraphrasing): "if the method is more interesting than the trick, the trick is not very good ... and that's when we show you the method. But the tricks we actually want to fool you with are just a bit of gaffer tape and a lie."
The fact that you say that you appreciate magic tricks that involve a lot of sleight of hand, or require technical mastery, kind of supports what we're saying. Most magic tricks aren't that.
To understand why, look at it from the point of view of a professional magician trying to pay the bills.
I can do card manipulation. I have a card manipulation act that I perform. It is by far the most technically challenging routine that I do and it has literally taken me years to get to the point where I can execute it and perform it. It's closer to juggling than it is to what most magic is (though it's still magic because I'm producing playing cards and fans of playing cards at my fingertips in short sleeves).
Now imagine that the majority of my act was that type of trick. I have bills to pay. I have crowds to entertain TODAY... and if I make just one mistake in a performance, the trick falls flat.
Penn & Teller tip cups & balls because the methods employed in that trick are genuinely entertaining to watch.
And the "let's have the audience lie"... I'm pretty sure you're referencing a segment in their 1990 TV special "Don't Try This at Home" where they had a large semi trailer truck run over Teller while surrounded by an audience who could see how it was done, which they tipped at the end. I wouldn't call that "lazy", I would call that using a medium to convey a message.
Magic doesn't work on television. You need to see it in person to really appreciate that it wasn't done with stooges/actors and camera tricks. That's a thesis that they have carried with them over the years. No one would do that "trick" in any other venue than in a television special. You can't get a big truck up on stage... and if you're doing "instant stooge" type of work, where people on stage from the audience are in on it ... at the risk of getting philosophical, they didn't even experience a magic trick. So it's not so much that the method was "lazy" ... it's that it was completely ineffective, and thus not even a magic trick for a certain group of people. Which was the entire point of the segment.
You're correct about the TV special with the truck. I don't agree that magic doesn't work on TV. Regardless of whether it's a TV show or performed in front of me the nature of magic† (as a performance) is that the performer gets to decide what to show and obviously they aren't going to leave only a single possibility open to explain what I saw, that's something Penn has talked about - there's only one way to do that trick in The Prestige so you would never do that trick.
I've never seen Penn & Teller live, but I have seen Derren Brown both on TV and in person. Now, on TV one of the things which most impresses me is Derren's use of forces. There are a handful that even an amateur who knows what they're looking for can see in some of the TV shows and there are more places where it's obviously a force but I can't figure out how it's done. In person though, that experience is actually less fun, because of course the force doesn't work on me. So he's forced a theatre full of other people to do what he wanted, and they don't know how. It didn't work on me in the TV audience but that's fine, I'm not the target - when it doesn't work in a theatre full of people it's a bit disappointing. When I listened to a recording of "Thou shalt always kill" I knew from the outset what the last two lines would be but that still kinda works, again in person it wouldn't land the same.
† If you claim not to be a magician, that this is real, then it's not up to you what is shown or not shown, if Millikan's drop experiment only worked with this custom made box and a specified oil recipe then it's just a trick. We do the experiment in whatever circumstances are available and it works because it's measuring a fact about our universe, it's not a trick.
I think what you mean by "force" is what magicians call a "psychological force." This is when, through suggestion, the spectator will name or do something that the magician was trying to get them to name/do, while having it look like the spectator had a free choice, through no other means than applied suggestion. These do "work", but as you pointed out they are not sure fire. You get the best results when you can do it on a large group of people and you're upfront about the fact that it won't work for everyone.
Although in Derren Brown's case, one of the things that makes him a genius in the world of magic is that he reversed that dynamic. One of the brilliant things about Derren Brown's "brand" of magic is that by blending stage hypnotism and parlour tricks, he does a lot of things where when you think you're watching a demonstration of stage hypnotism, it's actually a parlour trick that is dressed up to look like stage hypnotism.
In other words, he might use a classic sleight of hand trick to do a "force" while finding a way to frame and present it as if he were using suggestion.
But stage hypnotism and suggestion .. .that's not magic tricks. So if Derren Brown is employing a psychological force, or doing a demonstration of suggestion that will work for the majority of the audience but not all ... that's not a magic trick. By definition, it is not a trick.
What magicians mean when they talk about "forces" are ways to make it look like the spectator, ANY spectator, had a free choice but it was engineered. A card force, for example, where you think you have a free choice of any playing card but the magician "forced" a specific card on you. Those are 100% guaranteed. They don't use suggestion, they use sleight of hand, and there is no person in the world that those don't work 100% of the time on unless the magician makes a mistake.
Back to Derren Brown, although him and Teller are close personal friends and they don't hate each other by any means... P&T have been very outspoken about the fact that they consider some of what Derren does to be unethical, because he presents magic tricks as if they are displays of stage hypnotism or suggestion. Like most "mentalists" he has flirted with a line that many magicians will not cross. Where he claims that by reading body language, or using applied suggestion etc. that he can do or know certain things ... when the reality is those things are executed via traditional parlour trick methods are often just classic magic tricks that most magicians know and can do but are presented with a new/different layer of lies.
So why doesn't magic work on TV ... well, you gave the answer by invoking Derren Brown. If he is doing these "forces" (which to clarify I think you mean psychological "forces") and they work 10 out of 10 times on television but they don't work on 100% of live audience members .. well then you know why magic is weak (at best) on TV ... because all we need to do is go ask 52 people to think of a playing card, and guess that they're thinking of the 7 of Hearts (for example), and then just only air the one take where the guess was right.
There is a YouTube creator named James Hoffman who is a coffee barista. And he was on an episode of Derren Brown's Trick or Treat (if I remember those details correctly). His story of being on that show is very illuminating. He talks about how he was shown 2 different tricks, the first one fried his brain, the second one was "meh." For whatever reason they aired the 2nd one that was "meh", but cut in the reaction to the first one where he was like "OMG OMG that was so amazing."
Why would you trust what magicians tell you, when they say they don't do shit like that on television? This is why, as a huge fan of magic, I kind of hate televised / pre-recorded magic. I have no idea what they did during the editing process. Whereas live, there are so many more restrictions and the audience has a much clearer understanding and impression of what those restrictions are. But on tv... anything is possible through editing and vfx.
Derren used to have a TV show (I don't remember the name) where they actually show some times a live trick that was recorded for the show doesn't work. Because of course it can't always work. It's like those out-takes at the end of a Jackie Chan movie, two hours of this guy being flawless isn't real, here is the time he was a second too slow and ended up taking a kick to the head and his "enemies" all rush over to check he's OK.
I understand the distinction you're making but I think it's so narrow that these overlap in practice, there will always be a psychological element to controlling the volunteer. They could do anything and you don't want that because that's definitely not part of the show.
Derren has also done that "Only show the one where it worked" in a TV show but lampshaded it by also later showing the annoying hours of trying it and failing over and over. Again that's pretty interesting and it's a case where you could just fix it, but where's the fun in that? I think that was on a show about an actual scam pulled on gamblers, where you give somebody the false impression that you're picking winners reliably. That's not a trick, it's just a failure to consider the broader picture - every other participant knows you sometimes pick losers, but one "lucky" person does not and they're the victim.
> I understand the distinction you're making but I think it's so narrow that these overlap in practice
Entire books have been written about this, literally. So obviously we're taking short-cuts in a HN thread. Teller has said "all magic is a psychology experiment" so you're correct in that regard.
The point that I'm trying to make is that magicians rarely take risks in live performances. It is a job and there is a lot at stake. Obviously anything can, and does, happen, but there is a very important reason that magicians (not me, but magicians as a whole) distinguish between "psychological forces" and "forces."
It is that "forces" are considered to be sure fire... so near to 100% guaranteed that we consider them 100% guaranteed. Whereas "psychological forces" are well understood to be much less than 100% guaranteed.
Do magicians ever use psychological forces in practice? Sure. If, when it fails, there is an "out." Where if you saw the same trick performed several times, it would play out slightly differently each time. For example, let's say that I ask you to think of a card, and then I take a 1 in 52 odds guess at what you're thinking just in case I hit. And maybe I will slightly improve my odds by guessing at commonly thought of playing cards. Most of the time it's not going to hit, and that's fine because there's a fall back. But it's done so that in the event that it hits we have what looks like a bonefide miracle. Magicians do that occasionally.
But most of the time it's not that. Most of the time the magician is executing something that is considered to be close enough to 100% that no fallback ("outs" as we call them) is necessary.
Now obviously you could have some jerk physically grab the deck out of your hands or say to you verbally "I want the 8 of spades" when you extend the cards to have them remove one. That can happen, sure, and an experienced performer will be able to navigate the situation and turn it around "you see the thing is, I could guess what card you were thinking of ... 68% of men between the age of 24 and 35 name the 8 of spades ... we need a RANDOM card for this trick" etc. (And that example is valid, but traditional performer wisdom is once you've recovered get that jerk off the stage and move on as fast as possible because they've just proven themselves to be disruptive).
See, I won't say magic can't work on television. It can work, but it does rely on a setting where the home audience can be reasonably sure the film crew is not in cahoots with the performer. This is uncommon, but in some limited scenarios like magic tricks on America's Got Talent, or say P&T's Fool Us there is little reason to cheat with video tricks, as those won't help with the judges/P&T at all, plus you risk having members of the studio audience call out the producers over such trickery.
Things like TV specials on the other hand, yeah not so much.
That's a good point, and P&T have said as much when it comes to Fool Us. That having a live audience was integral to making the show work.
However, as much as I'm a fan of Fool Us, there are things that bother me about the show to the point where I will walk back how I phrased things. It's not that magic "can't work" on TV ... it's that it is always, without exception weaker on TV.
Fool Us is a show made by magicians for magicians (and I'm not even talking about Penn & Teller specifically... I'm talking about Johnny Thompson, Michael Close and others who have worked as producers to make that show what it is) and even they can't always adhere to not cutting during key moments because the nature of television forces them to have to fit varied material during a rigid time block that can change even up to moments before airing.
Then, if you pay attention when watching the show, they use other editing techniques like "risers" for example (a type of subtle sound fx added in post) during the recording which changes it in ways that makes it different than what you would experience if you saw it live.
And then there are still examples of doing stuff for TV with a live audience, that could only be done in that setting.
I'll give you an example of such a trick that has been done on these types of shows numerous times. The first time I saw it performed by Derren Brown on one of his shows I was completely brain-fried.
The trick is that the magician apparently "steals" a spectator's ability to read.
How it looks is that there is a word printed in large bold letters on a piece of paper (in Derren Brown's case it was printed on a page in a magazine). This word is shown to the wider audience and to the camera so that viewers can see it at home. What's great about this trick on TV is that while it is being shown to the selected audience member on stage, the camera can frame it so that viewers at home can verify that they are seeing the same printed word.
And yet while you can read it fine, the spectator on stage sees jibberish.
This trick has been performed on AGT at least once. It's also been performed on Fool Us by a different magician.
It's a marketed trick that you could go buy from a magic shop and perform yourself. But you won't. Why?
Because it works on optical principle that requires lighting to be very controlled.
The creator of that trick even recommends discussing the mechanics of the trick with the film production crew when doing it on television so that the camera operators and the director of photography know how to make it look as good as possible.
In other words... it's a great trick to perform on a sound stage when producing for television. But it doesn't really work that well in other environments. To be fair, you could do it with a live audience in a theatre and there are a lot of stage illusions that need controlled lighting. So it's not like this is something that needs TV specifically, though it kind of still needs cameras to "sell" the illusion so the audience can see what is being shown to the spectator on stage. And magicians love to have philosophical arguments about what constitutes "cheating." You've got magicians like me who think that the audience can't trust TV, and so we're advocates for live entertainment. And then you have magicians who take the position that TV is just another venue, and that the venue is always a tool that is available to be exploited and manipulated because that's what magicians do. But that leaves room for using everything from manipulative editing to cgi.
This is why, in my opinion, while I will walk back what I said a little and rephrase it as "magic is weaker on TV" rather than "can't work", I still think that magic is way more impressive in person... as a result of knowing what's possible on television, even when there is a live in-studio audience.
I've recently done reading "Digital Design and Computer Architecture" by Harris and Harris, and the part about microarchitecture had this exact impression on me: "oh, so we just demux the opcode, enable/disable the necessary signals/paths and it all... just works out in the end, if the clock is fine. Huh. Well, in retrospect it's kinda obvious."
> It's when you take apart a mechanical clock and keep looking for the time-keeping part, until you figure out that there isn't a time-keeping part in there, it's just gears and a spring.
The time-keeping part is arranging gears and a spring in a way that will, in fact, keep time.
> It's when you learn about integrated circuits and full-adders, and keep trying to understand how a bunch of transistors can do Mathematics, until you figure out that there isn't a mathematics-doing part in there, it's just circuits and wires, arranged in a way that makes the voltages come out right.
The mathematics-doing part in there is the arrangement of circuits and wires in ways that can actually do arithmetical operations on voltages.
It's not magic. But an adder, while never more than a bunch of circuits and wires, is still a mathematics-doing part.
Well to be pedantic: the time-keeping part isn’t the gears but a pendulum.
The spring gives energy to the pendulum, but that can’t effect more than in its amplitude: the period of a given pendulum is constant. Later springs demultiply the tick tack of the pendulum into desired units.
The heart of the clock is that choke on energy though a period.
Thats also why the famous phrase: clocks dont measure time but other clocks.
One interesting thing of (accidentally) starting with assembly is that you mostly can’t see magic at all, but instead you see learning magicians explaining computers in all sorts of funny ways.
My first “pc” was a clone of zx spectrum and all I had was a built-in BASIC and then some assembler on a cassette. Both went with “books” on how to use them, together with all of the unlimited time you have when you’re a kid.
This transferred to my first PC and eventually I learned how FAT, DOS, BIOS work, how to make a TSR and fool around B8000/A0000, first steps with 386+. It also helped that my granddad was an impulse electronics engineer and taught me how actual gates work and how computers count, sum and select numbers. He also had access to multiple books on hardware. I knew it all down to the silicon.
Other people had all sorts of magical ideas on how computers work. Special “hidden system areas”, “graphics card does X”, “computers multiply by addition”, etc etc. It’s a human thing that if we don’t understand something, our mind tries to yadda yadda it.
And the more you yadda yadda, the less chances it leaves that you’ll actually learn it. I tend to fight with these half-baked autogenerated explanations and try to dig down to how it really works. For no particular reason, that’s just what I like to do. It leaves a mark on how you work though.
I agree, hallucination is a completely different phenomena from inadvertently filling in knowledge gaps.
Hallucinating to me, is not a one off effect, but a dynamic phenomena. When our sensory processing/interpreting continues iterating, but wanders away from, or separates from, actual sensory input.
Dreams being a functional example. Drugs that cause our sensory systems to be disrupted or overwhelmed by unusual internal signals, being another example.
LLM’s operate in discrete steps. So that “interpretation continues iterating” is a very good description of what’s actually happening.
There’s a little uncertainty in the process, so sometimes it will pick the wrong symbol at the wrong time and the entire process just spirals into nonsense. But it’s semi lucid nonsense like a dream or hallucination not line noise.
The confidently stating the wrong thing bit is arguably a different though related problem. There’s making up a citations that don’t exist and there’s inserting song lyrics where a citation should be.
Confabulation: Inadvertent pseudo-memory fill in, within a generally reasonable stable context representation.
Hallucination: Context representation becomes unstable, resulting in open ended drifting and morphing into incoherence.
The scale of error and recursion of error in the latter make the effects quite different.
You could be right, perhaps sometimes the initial causes might not be so different. But I would be surprised if that was true, given there doesn't seem to be much of a middle ground between the wide differences in scale and downstream effect.
You could be right that it’s a fundamental difference in what’s going on, but each word is a new iteration/token and most errors are more than a single token.
I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. Where the boundary between the two states is arbitrarily small, but OpenAI etc has fine tuned the system so you generally see the converging / confabulation types of errors even if the difference is just a slight change in the scale of error.
> I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems.
That's a really good point.
Similar causes diverging in effect, a phase change or not, depending on whether a critical feedback threshold is hit or not. That could certainly be a factor.
With increasing doses of some drugs, humans do progress from being just a bit wonky, to full on lost in space!
But I think confabulation might be on a different continuum too. We fill in recalled memories with made up details all the time. Our memory storage is lossy, associative and overlapping. Our recall is always an imperfect combination of actual incidence memory filled out by similar memories.
We must confabulate to recall. Statistically, sometimes we over confabulate.
Hallucinate seems like a good word for image-producing neural nets (which is probably where it’s use originated?). Confabulate might be a better word for talkative LLMs, and less good for images.
I wonder if any neuroscientists or psychologists agree about humans and neural nets being similar this way. It’s less than unlikely that the mechanisms are the same between people and LLMs saying not true things. Aside from there being a wide variety of reasons people fabricate untrue things, we already know the mechanisms for neural net hallucinations and confabulations. It’s a non-self-aware machine designed to output tokens or pixels or whatever, it will turn the crank and spit out something with no concept of whether it’s true or not at all times. People, on the other hand, are often using emotion to drive what they say (and often without knowing it). People will sometimes rationalize their confabulations, sometimes say untrue things based on belief, sometimes say things driven by fear or embarrassment, sometimes lie because they have ulterior motives. None of these things apply to neural nets, so the similarity between human and NN confabulations seems at best limited to superficial summaries, no?
I was talking about the tendency to purely imagine things in low knowledge situation, but (iiuc) agree that this is a thing with our minds as well. Mine can skip a whole page of text if an earlier paragraph made it actively think. After that it’s sure we read it all. I have to detect that and rewind properly.
This (roughly) came up in a chat just yesterday. Someone posted a screenshot of a web browser using a font face with no glyphs or something, or I assume this is what causes this display of the codepoints(1). Reminded me of the kind of amateur magic lessons I see regularly:
So, people on the internet who feel like they are some kind of computer wizards will try to school each other online about how computers work. They will go on, like, Quora and ask weird questions like "what is one programming language every real hacker should know?" And someone will come along and write an incredibly definitive-sounding and confident essay about how a real hacker has to know the language of binary, because everything that's happening in a computer is all just binary, 1s and 0s. They heavily imply (at the least) that a real hacker regularly peruses binary values. Not like how you might use a hex editor. Rather, as if wannabe hackers read source, but real hackers read the raw binary. Some kind of M-x Butterfly shit. Assembly language is rarely mentioned.
I don't blame them for it, it's just kind of funny. I can kind of imagine that the way low-level things are glossed over for the sake of explaining high-level things has reasonably left some people with the impression that binary is somehow expressive and that people who truly know computers are fluent in it. And I probably had stranger conceptions than that on my own journey.
Anyway, this screenshot is what they must imagine a turbohacker's computer looks like all the time
Ah, this is the reason I avoid internet squares and chats filled with those who started but didn’t go deep. I just cringe to almost fainting everytime I read something there and my ears burn. Programmer’s humor sections are especially triggering. Surprisingly, HN is not that programmy at all (there’s lots of tech but no code nor “hacker secrets”), so it’s relatively nice to be in.
Lol, now for some reason I'm wondering what you think of the old "how to shoot yourself in the foot" / "how to determine which programming language you're using" jokes.
I guess one actually realistic version of the fully-jacked-in-hacker fantasy is network guys who are in so deep that they can read bin/hex numbers fluently without consciously "converting", etc. This I know of, and from there, I can conceive of someone who has become familiar with the binary representations of certain instructions? Or something? But I guess it would have to be a very specific (and probably read-only) situation. And, unless I'm missing something, it wouldn't amount to a deeper or more enlightened understanding of anything meaningful to software concerns
So-so, honestly. Was okay at the time. Sorry for not sharing it, but that’s how I feel.
Btw, reading and calculating hex is not that hard. In Hiew (and other hex/asm binary file editors that showed hex together with mnemonics) you quickly learn that e.g. B8 xx xx is “mov ax, imm” and C9/CB is a return. You can actually spot these in an 8-bit hex view, and yes, code has a pretty distinct feel in hex and ascii/exts. Zips and data don’t look like code.
For me hex code was read-only, but I remember my granddad entering programs through literally 8 on-off switches plus some more on a pre-sinclair “computer”. I could input hex values. He could input D0-D7 and #CLK itself.
That said there’s not much “hacker” in it. It’s just a knowledge induced by a relatively low level tooling and the times. We did what we did out of interest, not for social attitude. So yeah.
> I’m sure I’m not alone, in that each time you pull the curtain off a piece of ‘magic’, you have the same thought: Oooooh yeah. I mean, well duh.. how else would you do that? I can't believe I couldn't see it.
This is the great paradox of good ideas. The best ideas are obvious, but only in retrospect. You would be very unlikely to encounter the idea by randomly fumbling around, yet it feels so simple and obvious (and often even easy to implement!) after someone else points it out and you mull it over. Usually it requires looking in a different dimension than the one you were focused on rather than looking further along familiar dimensions, which is our default behavior.
In this way there’s a parallel to modern machine learning, where backprop on gigantic models allows us to find very short paths in very high dimensions, rather than finding very long circuitous paths in low dimensions. It turns out this also solves the overfitting problem, in the same way that “retrospectively obvious” is a good filter for ideas.
So many frameworks and dev-oriented features are sold with the claim that domain X is super complicated and full of esoteric useless trivia, so instead someone has put a clean facade on that which simplifies everything and protects the hapless dev from having to learn. Which is nice, it’s a tidy abstraction, we all have other things to do, now we go fast.
Except…the dev has to learn the new API, which we can call domain Y, and which can become quite complicated and hard to map conceptually to domain X (e.g. React events vs browser rendering, or Java GC vs real memory).
And when the cool facade doesn’t quite work for some exotic use case, now what? The dev gets to learn the original domain X anyway, plus how X and Y interact. In the worst case they have rewrite all their Y-using code.
Great abstractions are good magic; bad abstractions are evil magic. And yet good vs evil is often hard to tell apart when you pick up the problem.
The only areas I’ve ever seen that were truly super complicated were when synchronization is involved.
It’s so hard to wrap my head around how a CPU reorders instructions and I need to think very slowly and very carefully whenever I work with synchronization.
Same with distributed systems. I implemented a toy version of raft forever ago, but just wrapping my head around it took a month, and I still think I don’t really grasp it.
Sync is easy. Concurrency is hard. Partly b/c as humans we can't multitask.
CPU reordering instructions (and speculative execution) should be not be hard to understand. It tries to do as much work as it could before it has to stall due to lack of data
Synchronization is like a sub-problem of working with concurrency.
In this context they’re the same afaics
CPU reordering isn’t hard to understand, itself, but building systems that depend on it is (like building a time sharing OS… very difficult) because it’s difficult to reason about. The implications of it are hard to understand because, as you say, we’re not naturally apt at multitasking
Sometimes a remnant of the magic remains in the math, though, even after you understand why the math must work. The Burrows-Wheeler transform used in bzip2 is an example of this for me. I get why it works, but it just feels too damned convenient, almost as if this is one of the universe’s APIs, and you don’t get to look into the source code of this one to find out why such a simple maneuver is so effective on exactly the sort of data we like to compress.
My own mental picture of that makes it just feel very clever, rather than magical. If you imagine that addition is easy, but division is hard, it’s pretty easy to see how you can build fast integer multiplication out of addition, and create hard to reverse operations like key exchange.
Now replace “addition” with modular multiplication, and “division” with the logarithm, and you have the trick behind DH. Swap out addition for this weird operation that involves jumping around elliptic curves, and you have the trick behind ECDH (admittedly, the fact that that jumping around thing works as a group operation is still pretty magical).
What I like about this way of thinking about it is that you can fully implement the key exchange in terms of integer addition, and the fact that it lets you pass around secrets in a world without “division oracles” seems pretty mundane.
TFW the real wizards are the ones working in the strata just below the one I understand...
After nearly 30 years of persistent excavation, I'm finally at bedrock with registers, some assembly and embedded C.
Lifting my head up to find the mess we've created for graphical application development is admittedly disheartening, but also thrilling as there are ways to make it better.
There is at least one more layer, which lies behind the abstraction boundary the ISA presents. E.g. x86-64 specifies only 16 general purpose registers, but the Zen 4 has 224. Even as far back as Pentium II the CPU had more registers than the ISA.
You’re not even close to hitting rock bottom on explanations of the layers below until you find yourself comparing proposed theories of quantum gravity :-)
I don't think ignoring negligible effects means you're a significant distance from the bottom. You can semi-skim through quantum effects and still have enough detail to stretch your scale to the bottom.
Now I kind of want to read a story where it turns out the technology we use in our modern lives is actually very carefully hidden magic. Sort of like the opposite of (rot13 for book spoilers) gur fgrrefjbzna frevrf.
On the topic of how that implementation comptime in Go works: I've toyed with a similar idea for implementing constant folding/beta reduction — generate a temporary file with the relevant subset of definitions, insert the constant expression/function call you'd like to evaluate and... compile and run this file. It may not be the most performant thing to do but at least you get the correct semantics without writing a separate constexpr interpreter.
One of the cool things about being a kid is I didn't know how tf anything works. So I thought long and hard about how things work and just randomly guessing. When it turns out later there's a clear answer, it's very satisfying. But knowledge can also weigh down and discourage curiosity.
Charles Petzold's book "Code" was the book that revealed the magic behind computers for me. While I don't fully understand everything that happens with a computer, there's a confidence that I could if I needed to. It made tough problems feel solvable in a way I hadn't felt in other disciplines.
Going the other direction, towards higher levels of abstraction tends to strip away the magic too!
I recently write some robotics code using ROS, taking a step back I looked a the result and thought: Actually that is not much different conceptually from running K8s deployments coupled by Kafka.
When understanding a new "magic", there's this beautiful moment when you grok it, and the abstraction poofs away.
It's when you take apart a mechanical clock and keep looking for the time-keeping part, until you figure out that there isn't a time-keeping part in there, it's just gears and a spring.
It's when you learn about integrated circuits and full-adders, and keep trying to understand how a bunch of transistors can do Mathematics, until you figure out that there isn't a mathematics-doing part in there, it's just circuits and wires, arranged in a way that makes the voltages come out right.
It's when your understanding of the top-down structure snaps together with the bottom-up mechanics of the building blocks. There's no space left for the ghost in the machine to haunt, and you go "Oh. huh". I live for that moment.
I went through an EE degree instead of a CS degree in undergrad specifically so I could peel back this magic and really understand what's going on, to some detail, down to the electromagnetism level. It is indeed a very freeing feeling to be resting atop so many layers of abstractions, with the understanding that if you ever had to, you could go down to any one of them and kind of feel your way around.
I think for me the biggest magic-killing moment was when I realized that CPU clock cycles, timing trees, etc. were all just ways for us to shoehorn a fundamentally analog thing (how much voltage is in area X?) to a digital thing (does area X have enough voltage? yes? OK, call it a 1 and let's move on already!). Somehow to me that feels like the ultimate "leaky" abstraction, although of course decades and trillions of dollars have gone into making it ever more watertight for 99.99-several-more-9s% of the time. At least to the end user. Your mileage may vary if you're a TSMC researcher or something, of course.
I've always felt the whole binary/digital thing was one of the most clever bits of compromise with the real world I've ever seen.
You have this thing, you want to be able to translate its value into something useful. In this case, the amount of voltage in a circuit to a number. And you spend so much time trying to make sure the voltage level passed is rock solid, that your read is equally solid, etc. Until you realize that you'd have to invent so many more industries just to do this one thing that you just give up and say the only thing you can know with certainty is that there is or is not voltage passing through the circuit.
Then you need to be able to translate "ON" and "OFF" into actual usable values. And eventually coming down to a base 2 counting system so that 4 circuits gives you 16 distinct values seems obvious in hindsight, but had to be a revelation when they realized it.
Bingo, precisely. I keep using this compromise as a foundational example when I talk to my wife (currently doing a CS degree) about why binary stuff, specifically, keeps coming up in her stuff. 2 is a magic number in computing because it is the Great Compromise we made with reality to get it.
Every now and then people here bring up more exotic paradigms, like ternary computing, and yes! In an ideal world the more phases you can detect the 'better', all the way up to infinity (pure analogue computing). But the difficulty curve to scale anything besides base 2 to where base 2 computing currently is is, at least for our current understanding in physics, materials science, etc., way higher.
Actually, Leibniz came up with the dual/binary system as being ideal for computation 300 years ago roughly around the same time he built his mechanical calculator, long before there was any chance of there being programmable computers.
I work with FPGAs and embedded systems occasionally, and you have no idea how amazingly watertight the phone/desktop/server CPU abstraction is in comparison to what you get the moment you do something slightly weird. A combination of the chips, the firmware, and the OS does so much work to give you the abstraction of "machine runs code and it just works."
There's a paper from the 1990s of someone trying to do neural net-type things to construct an FPGA, and it gets really out there, like 'this gate isn't actually connected to anything but if we change anything about it the laws of electromagnetism cause the whole thing to stop working' weird. FPGAs are fascinating little dudes, esp in domains like HFT where their speed directly translates into dollars, but I totally believe you here.
https://www.damninteresting.com/on-the-origin-of-circuits/
for me, it was latches and creating a CPU which instructions could be fed into. but it wasn't a "lost the magic" feeling, it was a "that's amazing!" feeling.
“The true delight is in the finding out rather than in the knowing.” ― Isaac Asimov
Keeping true to your username :)
Man I'm not going to lie tho... I just could not make it through the foundation series
While the Foundation series is what's best known nowadays, the Suzan Calvin and Elijah Bailay stuff, and more generally everything set before the Galactic Empire even exists in Asimov's fictional timeline is where Asimov is at his best.
I liked them. But I was pretty young when I read them, so pretty much any book had new insights and wisdom to offer me.
That's the difference between technology magic and illusionist magic, When you see how the trick is done with illusions it's always a bit of a letdown because the answer is usually prosaic, the 'magic' vanishes and it becomes a trick.
When you understand how a piece of technology works you get that beautiful moment.
I've never got that. I feel the same way in either case, if your trick was easy everybody would do it. Sleight of hand tricks for example, if you're good they're completely seamless, I could never hope to reproduce and yet I know exactly how it's done.
Take that Penn & Teller trick where the live audience is just lying - that's a bit lazy, we're not supposed to have some great admiration for this trick they're just showing you it's an option to just have the live audience lie to the recorded audience and that "works". Whereas their transparent version of cups and balls is the opposite, you must be a sleight of hand master or this won't look like anything.
My wife and I are part time magicians. In my experience, people who share your mindset become magicians.
At the risk of just repeating exactly what the person you're replying to wrote: The reason that non-magicians often feel let down when they find out how a trick is executed, is that it often feels like an insult to their intelligence. Magic toys with and exploits your assumptions. As soon as you find out that those assumptions were incorrect, and that you were just lied to by a prop (for example), your experience and perception of that illusion goes from "OMG" to "oh, that's all?"
What we want the methods to be is some grandiose Ocean's 11 like "heist" with tons of sneaky maneuvers and difficult sleight of hand ... but MOST magic tricks, including the ones that Penn & Teller fool you with (not the ones where they tip the method) are not that.
Penn & Teller have said themselves that (paraphrasing): "if the method is more interesting than the trick, the trick is not very good ... and that's when we show you the method. But the tricks we actually want to fool you with are just a bit of gaffer tape and a lie."
The fact that you say that you appreciate magic tricks that involve a lot of sleight of hand, or require technical mastery, kind of supports what we're saying. Most magic tricks aren't that.
To understand why, look at it from the point of view of a professional magician trying to pay the bills.
I can do card manipulation. I have a card manipulation act that I perform. It is by far the most technically challenging routine that I do and it has literally taken me years to get to the point where I can execute it and perform it. It's closer to juggling than it is to what most magic is (though it's still magic because I'm producing playing cards and fans of playing cards at my fingertips in short sleeves).
Now imagine that the majority of my act was that type of trick. I have bills to pay. I have crowds to entertain TODAY... and if I make just one mistake in a performance, the trick falls flat.
Penn & Teller tip cups & balls because the methods employed in that trick are genuinely entertaining to watch.
And the "let's have the audience lie"... I'm pretty sure you're referencing a segment in their 1990 TV special "Don't Try This at Home" where they had a large semi trailer truck run over Teller while surrounded by an audience who could see how it was done, which they tipped at the end. I wouldn't call that "lazy", I would call that using a medium to convey a message.
Magic doesn't work on television. You need to see it in person to really appreciate that it wasn't done with stooges/actors and camera tricks. That's a thesis that they have carried with them over the years. No one would do that "trick" in any other venue than in a television special. You can't get a big truck up on stage... and if you're doing "instant stooge" type of work, where people on stage from the audience are in on it ... at the risk of getting philosophical, they didn't even experience a magic trick. So it's not so much that the method was "lazy" ... it's that it was completely ineffective, and thus not even a magic trick for a certain group of people. Which was the entire point of the segment.
You're correct about the TV special with the truck. I don't agree that magic doesn't work on TV. Regardless of whether it's a TV show or performed in front of me the nature of magic† (as a performance) is that the performer gets to decide what to show and obviously they aren't going to leave only a single possibility open to explain what I saw, that's something Penn has talked about - there's only one way to do that trick in The Prestige so you would never do that trick.
I've never seen Penn & Teller live, but I have seen Derren Brown both on TV and in person. Now, on TV one of the things which most impresses me is Derren's use of forces. There are a handful that even an amateur who knows what they're looking for can see in some of the TV shows and there are more places where it's obviously a force but I can't figure out how it's done. In person though, that experience is actually less fun, because of course the force doesn't work on me. So he's forced a theatre full of other people to do what he wanted, and they don't know how. It didn't work on me in the TV audience but that's fine, I'm not the target - when it doesn't work in a theatre full of people it's a bit disappointing. When I listened to a recording of "Thou shalt always kill" I knew from the outset what the last two lines would be but that still kinda works, again in person it wouldn't land the same.
† If you claim not to be a magician, that this is real, then it's not up to you what is shown or not shown, if Millikan's drop experiment only worked with this custom made box and a specified oil recipe then it's just a trick. We do the experiment in whatever circumstances are available and it works because it's measuring a fact about our universe, it's not a trick.
I think what you mean by "force" is what magicians call a "psychological force." This is when, through suggestion, the spectator will name or do something that the magician was trying to get them to name/do, while having it look like the spectator had a free choice, through no other means than applied suggestion. These do "work", but as you pointed out they are not sure fire. You get the best results when you can do it on a large group of people and you're upfront about the fact that it won't work for everyone.
Although in Derren Brown's case, one of the things that makes him a genius in the world of magic is that he reversed that dynamic. One of the brilliant things about Derren Brown's "brand" of magic is that by blending stage hypnotism and parlour tricks, he does a lot of things where when you think you're watching a demonstration of stage hypnotism, it's actually a parlour trick that is dressed up to look like stage hypnotism.
In other words, he might use a classic sleight of hand trick to do a "force" while finding a way to frame and present it as if he were using suggestion.
But stage hypnotism and suggestion .. .that's not magic tricks. So if Derren Brown is employing a psychological force, or doing a demonstration of suggestion that will work for the majority of the audience but not all ... that's not a magic trick. By definition, it is not a trick.
What magicians mean when they talk about "forces" are ways to make it look like the spectator, ANY spectator, had a free choice but it was engineered. A card force, for example, where you think you have a free choice of any playing card but the magician "forced" a specific card on you. Those are 100% guaranteed. They don't use suggestion, they use sleight of hand, and there is no person in the world that those don't work 100% of the time on unless the magician makes a mistake.
Back to Derren Brown, although him and Teller are close personal friends and they don't hate each other by any means... P&T have been very outspoken about the fact that they consider some of what Derren does to be unethical, because he presents magic tricks as if they are displays of stage hypnotism or suggestion. Like most "mentalists" he has flirted with a line that many magicians will not cross. Where he claims that by reading body language, or using applied suggestion etc. that he can do or know certain things ... when the reality is those things are executed via traditional parlour trick methods are often just classic magic tricks that most magicians know and can do but are presented with a new/different layer of lies.
So why doesn't magic work on TV ... well, you gave the answer by invoking Derren Brown. If he is doing these "forces" (which to clarify I think you mean psychological "forces") and they work 10 out of 10 times on television but they don't work on 100% of live audience members .. well then you know why magic is weak (at best) on TV ... because all we need to do is go ask 52 people to think of a playing card, and guess that they're thinking of the 7 of Hearts (for example), and then just only air the one take where the guess was right.
There is a YouTube creator named James Hoffman who is a coffee barista. And he was on an episode of Derren Brown's Trick or Treat (if I remember those details correctly). His story of being on that show is very illuminating. He talks about how he was shown 2 different tricks, the first one fried his brain, the second one was "meh." For whatever reason they aired the 2nd one that was "meh", but cut in the reaction to the first one where he was like "OMG OMG that was so amazing."
Why would you trust what magicians tell you, when they say they don't do shit like that on television? This is why, as a huge fan of magic, I kind of hate televised / pre-recorded magic. I have no idea what they did during the editing process. Whereas live, there are so many more restrictions and the audience has a much clearer understanding and impression of what those restrictions are. But on tv... anything is possible through editing and vfx.
Derren used to have a TV show (I don't remember the name) where they actually show some times a live trick that was recorded for the show doesn't work. Because of course it can't always work. It's like those out-takes at the end of a Jackie Chan movie, two hours of this guy being flawless isn't real, here is the time he was a second too slow and ended up taking a kick to the head and his "enemies" all rush over to check he's OK.
I understand the distinction you're making but I think it's so narrow that these overlap in practice, there will always be a psychological element to controlling the volunteer. They could do anything and you don't want that because that's definitely not part of the show.
Derren has also done that "Only show the one where it worked" in a TV show but lampshaded it by also later showing the annoying hours of trying it and failing over and over. Again that's pretty interesting and it's a case where you could just fix it, but where's the fun in that? I think that was on a show about an actual scam pulled on gamblers, where you give somebody the false impression that you're picking winners reliably. That's not a trick, it's just a failure to consider the broader picture - every other participant knows you sometimes pick losers, but one "lucky" person does not and they're the victim.
> I understand the distinction you're making but I think it's so narrow that these overlap in practice
Entire books have been written about this, literally. So obviously we're taking short-cuts in a HN thread. Teller has said "all magic is a psychology experiment" so you're correct in that regard.
The point that I'm trying to make is that magicians rarely take risks in live performances. It is a job and there is a lot at stake. Obviously anything can, and does, happen, but there is a very important reason that magicians (not me, but magicians as a whole) distinguish between "psychological forces" and "forces."
It is that "forces" are considered to be sure fire... so near to 100% guaranteed that we consider them 100% guaranteed. Whereas "psychological forces" are well understood to be much less than 100% guaranteed.
Do magicians ever use psychological forces in practice? Sure. If, when it fails, there is an "out." Where if you saw the same trick performed several times, it would play out slightly differently each time. For example, let's say that I ask you to think of a card, and then I take a 1 in 52 odds guess at what you're thinking just in case I hit. And maybe I will slightly improve my odds by guessing at commonly thought of playing cards. Most of the time it's not going to hit, and that's fine because there's a fall back. But it's done so that in the event that it hits we have what looks like a bonefide miracle. Magicians do that occasionally.
But most of the time it's not that. Most of the time the magician is executing something that is considered to be close enough to 100% that no fallback ("outs" as we call them) is necessary.
Now obviously you could have some jerk physically grab the deck out of your hands or say to you verbally "I want the 8 of spades" when you extend the cards to have them remove one. That can happen, sure, and an experienced performer will be able to navigate the situation and turn it around "you see the thing is, I could guess what card you were thinking of ... 68% of men between the age of 24 and 35 name the 8 of spades ... we need a RANDOM card for this trick" etc. (And that example is valid, but traditional performer wisdom is once you've recovered get that jerk off the stage and move on as fast as possible because they've just proven themselves to be disruptive).
See, I won't say magic can't work on television. It can work, but it does rely on a setting where the home audience can be reasonably sure the film crew is not in cahoots with the performer. This is uncommon, but in some limited scenarios like magic tricks on America's Got Talent, or say P&T's Fool Us there is little reason to cheat with video tricks, as those won't help with the judges/P&T at all, plus you risk having members of the studio audience call out the producers over such trickery.
Things like TV specials on the other hand, yeah not so much.
That's a good point, and P&T have said as much when it comes to Fool Us. That having a live audience was integral to making the show work.
However, as much as I'm a fan of Fool Us, there are things that bother me about the show to the point where I will walk back how I phrased things. It's not that magic "can't work" on TV ... it's that it is always, without exception weaker on TV.
Fool Us is a show made by magicians for magicians (and I'm not even talking about Penn & Teller specifically... I'm talking about Johnny Thompson, Michael Close and others who have worked as producers to make that show what it is) and even they can't always adhere to not cutting during key moments because the nature of television forces them to have to fit varied material during a rigid time block that can change even up to moments before airing.
Then, if you pay attention when watching the show, they use other editing techniques like "risers" for example (a type of subtle sound fx added in post) during the recording which changes it in ways that makes it different than what you would experience if you saw it live.
And then there are still examples of doing stuff for TV with a live audience, that could only be done in that setting.
I'll give you an example of such a trick that has been done on these types of shows numerous times. The first time I saw it performed by Derren Brown on one of his shows I was completely brain-fried.
The trick is that the magician apparently "steals" a spectator's ability to read.
How it looks is that there is a word printed in large bold letters on a piece of paper (in Derren Brown's case it was printed on a page in a magazine). This word is shown to the wider audience and to the camera so that viewers can see it at home. What's great about this trick on TV is that while it is being shown to the selected audience member on stage, the camera can frame it so that viewers at home can verify that they are seeing the same printed word.
And yet while you can read it fine, the spectator on stage sees jibberish.
This trick has been performed on AGT at least once. It's also been performed on Fool Us by a different magician.
It's a marketed trick that you could go buy from a magic shop and perform yourself. But you won't. Why?
Because it works on optical principle that requires lighting to be very controlled.
The creator of that trick even recommends discussing the mechanics of the trick with the film production crew when doing it on television so that the camera operators and the director of photography know how to make it look as good as possible.
In other words... it's a great trick to perform on a sound stage when producing for television. But it doesn't really work that well in other environments. To be fair, you could do it with a live audience in a theatre and there are a lot of stage illusions that need controlled lighting. So it's not like this is something that needs TV specifically, though it kind of still needs cameras to "sell" the illusion so the audience can see what is being shown to the spectator on stage. And magicians love to have philosophical arguments about what constitutes "cheating." You've got magicians like me who think that the audience can't trust TV, and so we're advocates for live entertainment. And then you have magicians who take the position that TV is just another venue, and that the venue is always a tool that is available to be exploited and manipulated because that's what magicians do. But that leaves room for using everything from manipulative editing to cgi.
This is why, in my opinion, while I will walk back what I said a little and rephrase it as "magic is weaker on TV" rather than "can't work", I still think that magic is way more impressive in person... as a result of knowing what's possible on television, even when there is a live in-studio audience.
I've recently done reading "Digital Design and Computer Architecture" by Harris and Harris, and the part about microarchitecture had this exact impression on me: "oh, so we just demux the opcode, enable/disable the necessary signals/paths and it all... just works out in the end, if the clock is fine. Huh. Well, in retrospect it's kinda obvious."
> It's when you take apart a mechanical clock and keep looking for the time-keeping part, until you figure out that there isn't a time-keeping part in there, it's just gears and a spring.
The time-keeping part is arranging gears and a spring in a way that will, in fact, keep time.
> It's when you learn about integrated circuits and full-adders, and keep trying to understand how a bunch of transistors can do Mathematics, until you figure out that there isn't a mathematics-doing part in there, it's just circuits and wires, arranged in a way that makes the voltages come out right.
The mathematics-doing part in there is the arrangement of circuits and wires in ways that can actually do arithmetical operations on voltages.
It's not magic. But an adder, while never more than a bunch of circuits and wires, is still a mathematics-doing part.
Well to be pedantic: the time-keeping part isn’t the gears but a pendulum.
The spring gives energy to the pendulum, but that can’t effect more than in its amplitude: the period of a given pendulum is constant. Later springs demultiply the tick tack of the pendulum into desired units.
The heart of the clock is that choke on energy though a period.
Thats also why the famous phrase: clocks dont measure time but other clocks.
(Please dont mind the grammar: writing on mobile)
Or a balance wheel
I was certain that you were going to conclude with a paragraph about LLMs.
I was this close to concluding with a paragraph about Buddhism and the Self. Which is basically the same thing, but from the first-person perspective.
One interesting thing of (accidentally) starting with assembly is that you mostly can’t see magic at all, but instead you see learning magicians explaining computers in all sorts of funny ways.
My first “pc” was a clone of zx spectrum and all I had was a built-in BASIC and then some assembler on a cassette. Both went with “books” on how to use them, together with all of the unlimited time you have when you’re a kid.
This transferred to my first PC and eventually I learned how FAT, DOS, BIOS work, how to make a TSR and fool around B8000/A0000, first steps with 386+. It also helped that my granddad was an impulse electronics engineer and taught me how actual gates work and how computers count, sum and select numbers. He also had access to multiple books on hardware. I knew it all down to the silicon.
Other people had all sorts of magical ideas on how computers work. Special “hidden system areas”, “graphics card does X”, “computers multiply by addition”, etc etc. It’s a human thing that if we don’t understand something, our mind tries to yadda yadda it.
And the more you yadda yadda, the less chances it leaves that you’ll actually learn it. I tend to fight with these half-baked autogenerated explanations and try to dig down to how it really works. For no particular reason, that’s just what I like to do. It leaves a mark on how you work though.
> yadda yadda
Like LLMs, we confabulate. We fill in knowledge holes, with imagined knowledge we don't really have.
> Like LLMs, we confabulate.
Thank you for describing LLM behaviour that way. It's a much better description than the more popular but less apt "hallucination".
I agree, hallucination is a completely different phenomena from inadvertently filling in knowledge gaps.
Hallucinating to me, is not a one off effect, but a dynamic phenomena. When our sensory processing/interpreting continues iterating, but wanders away from, or separates from, actual sensory input.
Dreams being a functional example. Drugs that cause our sensory systems to be disrupted or overwhelmed by unusual internal signals, being another example.
LLM’s operate in discrete steps. So that “interpretation continues iterating” is a very good description of what’s actually happening.
There’s a little uncertainty in the process, so sometimes it will pick the wrong symbol at the wrong time and the entire process just spirals into nonsense. But it’s semi lucid nonsense like a dream or hallucination not line noise.
The confidently stating the wrong thing bit is arguably a different though related problem. There’s making up a citations that don’t exist and there’s inserting song lyrics where a citation should be.
Confabulation: Inadvertent pseudo-memory fill in, within a generally reasonable stable context representation.
Hallucination: Context representation becomes unstable, resulting in open ended drifting and morphing into incoherence.
The scale of error and recursion of error in the latter make the effects quite different.
You could be right, perhaps sometimes the initial causes might not be so different. But I would be surprised if that was true, given there doesn't seem to be much of a middle ground between the wide differences in scale and downstream effect.
You could be right that it’s a fundamental difference in what’s going on, but each word is a new iteration/token and most errors are more than a single token.
I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. Where the boundary between the two states is arbitrarily small, but OpenAI etc has fine tuned the system so you generally see the converging / confabulation types of errors even if the difference is just a slight change in the scale of error.
> I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems.
That's a really good point.
Similar causes diverging in effect, a phase change or not, depending on whether a critical feedback threshold is hit or not. That could certainly be a factor.
With increasing doses of some drugs, humans do progress from being just a bit wonky, to full on lost in space!
But I think confabulation might be on a different continuum too. We fill in recalled memories with made up details all the time. Our memory storage is lossy, associative and overlapping. Our recall is always an imperfect combination of actual incidence memory filled out by similar memories.
We must confabulate to recall. Statistically, sometimes we over confabulate.
You're in good company with that thought. Many deep learning researchers and practitioners have suggested the same thing.
Heck, even ChatGPT itself agrees: https://community.openai.com/t/hallucination-vs-confabulatio...
Hallucinate seems like a good word for image-producing neural nets (which is probably where it’s use originated?). Confabulate might be a better word for talkative LLMs, and less good for images.
I wonder if any neuroscientists or psychologists agree about humans and neural nets being similar this way. It’s less than unlikely that the mechanisms are the same between people and LLMs saying not true things. Aside from there being a wide variety of reasons people fabricate untrue things, we already know the mechanisms for neural net hallucinations and confabulations. It’s a non-self-aware machine designed to output tokens or pixels or whatever, it will turn the crank and spit out something with no concept of whether it’s true or not at all times. People, on the other hand, are often using emotion to drive what they say (and often without knowing it). People will sometimes rationalize their confabulations, sometimes say untrue things based on belief, sometimes say things driven by fear or embarrassment, sometimes lie because they have ulterior motives. None of these things apply to neural nets, so the similarity between human and NN confabulations seems at best limited to superficial summaries, no?
What the GP is explaining is actually the other way around. It's our brain leaving holes even though they are just explained on the text.
And yeah ML models do that too.
I was talking about the tendency to purely imagine things in low knowledge situation, but (iiuc) agree that this is a thing with our minds as well. Mine can skip a whole page of text if an earlier paragraph made it actively think. After that it’s sure we read it all. I have to detect that and rewind properly.
This (roughly) came up in a chat just yesterday. Someone posted a screenshot of a web browser using a font face with no glyphs or something, or I assume this is what causes this display of the codepoints(1). Reminded me of the kind of amateur magic lessons I see regularly:
So, people on the internet who feel like they are some kind of computer wizards will try to school each other online about how computers work. They will go on, like, Quora and ask weird questions like "what is one programming language every real hacker should know?" And someone will come along and write an incredibly definitive-sounding and confident essay about how a real hacker has to know the language of binary, because everything that's happening in a computer is all just binary, 1s and 0s. They heavily imply (at the least) that a real hacker regularly peruses binary values. Not like how you might use a hex editor. Rather, as if wannabe hackers read source, but real hackers read the raw binary. Some kind of M-x Butterfly shit. Assembly language is rarely mentioned.
I don't blame them for it, it's just kind of funny. I can kind of imagine that the way low-level things are glossed over for the sake of explaining high-level things has reasonably left some people with the impression that binary is somehow expressive and that people who truly know computers are fluent in it. And I probably had stranger conceptions than that on my own journey.
Anyway, this screenshot is what they must imagine a turbohacker's computer looks like all the time
1. https://i.vgy.me/qNU6J5.jpg
Ah, this is the reason I avoid internet squares and chats filled with those who started but didn’t go deep. I just cringe to almost fainting everytime I read something there and my ears burn. Programmer’s humor sections are especially triggering. Surprisingly, HN is not that programmy at all (there’s lots of tech but no code nor “hacker secrets”), so it’s relatively nice to be in.
Lol, now for some reason I'm wondering what you think of the old "how to shoot yourself in the foot" / "how to determine which programming language you're using" jokes.
I guess one actually realistic version of the fully-jacked-in-hacker fantasy is network guys who are in so deep that they can read bin/hex numbers fluently without consciously "converting", etc. This I know of, and from there, I can conceive of someone who has become familiar with the binary representations of certain instructions? Or something? But I guess it would have to be a very specific (and probably read-only) situation. And, unless I'm missing something, it wouldn't amount to a deeper or more enlightened understanding of anything meaningful to software concerns
how to … jokes
So-so, honestly. Was okay at the time. Sorry for not sharing it, but that’s how I feel.
Btw, reading and calculating hex is not that hard. In Hiew (and other hex/asm binary file editors that showed hex together with mnemonics) you quickly learn that e.g. B8 xx xx is “mov ax, imm” and C9/CB is a return. You can actually spot these in an 8-bit hex view, and yes, code has a pretty distinct feel in hex and ascii/exts. Zips and data don’t look like code.
For me hex code was read-only, but I remember my granddad entering programs through literally 8 on-off switches plus some more on a pre-sinclair “computer”. I could input hex values. He could input D0-D7 and #CLK itself.
That said there’s not much “hacker” in it. It’s just a knowledge induced by a relatively low level tooling and the times. We did what we did out of interest, not for social attitude. So yeah.
> Special “hidden system areas”
I guess "hidden" can be a matter of perspective, from filenames beginning with a period, right through to things like the Intel Management Engine.
> I’m sure I’m not alone, in that each time you pull the curtain off a piece of ‘magic’, you have the same thought: Oooooh yeah. I mean, well duh.. how else would you do that? I can't believe I couldn't see it.
This is the great paradox of good ideas. The best ideas are obvious, but only in retrospect. You would be very unlikely to encounter the idea by randomly fumbling around, yet it feels so simple and obvious (and often even easy to implement!) after someone else points it out and you mull it over. Usually it requires looking in a different dimension than the one you were focused on rather than looking further along familiar dimensions, which is our default behavior.
In this way there’s a parallel to modern machine learning, where backprop on gigantic models allows us to find very short paths in very high dimensions, rather than finding very long circuitous paths in low dimensions. It turns out this also solves the overfitting problem, in the same way that “retrospectively obvious” is a good filter for ideas.
So many frameworks and dev-oriented features are sold with the claim that domain X is super complicated and full of esoteric useless trivia, so instead someone has put a clean facade on that which simplifies everything and protects the hapless dev from having to learn. Which is nice, it’s a tidy abstraction, we all have other things to do, now we go fast.
Except…the dev has to learn the new API, which we can call domain Y, and which can become quite complicated and hard to map conceptually to domain X (e.g. React events vs browser rendering, or Java GC vs real memory).
And when the cool facade doesn’t quite work for some exotic use case, now what? The dev gets to learn the original domain X anyway, plus how X and Y interact. In the worst case they have rewrite all their Y-using code.
Great abstractions are good magic; bad abstractions are evil magic. And yet good vs evil is often hard to tell apart when you pick up the problem.
The only areas I’ve ever seen that were truly super complicated were when synchronization is involved.
It’s so hard to wrap my head around how a CPU reorders instructions and I need to think very slowly and very carefully whenever I work with synchronization.
Same with distributed systems. I implemented a toy version of raft forever ago, but just wrapping my head around it took a month, and I still think I don’t really grasp it.
Haven’t even looked into paxos yet.
Sync is easy. Concurrency is hard. Partly b/c as humans we can't multitask.
CPU reordering instructions (and speculative execution) should be not be hard to understand. It tries to do as much work as it could before it has to stall due to lack of data
Synchronization is like a sub-problem of working with concurrency.
In this context they’re the same afaics
CPU reordering isn’t hard to understand, itself, but building systems that depend on it is (like building a time sharing OS… very difficult) because it’s difficult to reason about. The implications of it are hard to understand because, as you say, we’re not naturally apt at multitasking
Sometimes a remnant of the magic remains in the math, though, even after you understand why the math must work. The Burrows-Wheeler transform used in bzip2 is an example of this for me. I get why it works, but it just feels too damned convenient, almost as if this is one of the universe’s APIs, and you don’t get to look into the source code of this one to find out why such a simple maneuver is so effective on exactly the sort of data we like to compress.
I've always thought the Diffie-Hellman key exchange is pretty neat.
My own mental picture of that makes it just feel very clever, rather than magical. If you imagine that addition is easy, but division is hard, it’s pretty easy to see how you can build fast integer multiplication out of addition, and create hard to reverse operations like key exchange.
Now replace “addition” with modular multiplication, and “division” with the logarithm, and you have the trick behind DH. Swap out addition for this weird operation that involves jumping around elliptic curves, and you have the trick behind ECDH (admittedly, the fact that that jumping around thing works as a group operation is still pretty magical).
What I like about this way of thinking about it is that you can fully implement the key exchange in terms of integer addition, and the fact that it lets you pass around secrets in a world without “division oracles” seems pretty mundane.
TFW the real wizards are the ones working in the strata just below the one I understand...
After nearly 30 years of persistent excavation, I'm finally at bedrock with registers, some assembly and embedded C.
Lifting my head up to find the mess we've created for graphical application development is admittedly disheartening, but also thrilling as there are ways to make it better.
There is at least one more layer, which lies behind the abstraction boundary the ISA presents. E.g. x86-64 specifies only 16 general purpose registers, but the Zen 4 has 224. Even as far back as Pentium II the CPU had more registers than the ISA.
You’re not even close to hitting rock bottom on explanations of the layers below until you find yourself comparing proposed theories of quantum gravity :-)
It is around that point that you may be convinced that rocks are capable of conscious thought and the strange loop begins again at the top.
I am a conscious thinking rock.
(kinda)
I don't think ignoring negligible effects means you're a significant distance from the bottom. You can semi-skim through quantum effects and still have enough detail to stretch your scale to the bottom.
Corollary: Sufficiently well encapsulated magic is indistinguishable from technology.
Now I kind of want to read a story where it turns out the technology we use in our modern lives is actually very carefully hidden magic. Sort of like the opposite of (rot13 for book spoilers) gur fgrrefjbzna frevrf.
The only two examples of real "magic" I've encountered (would be interested in more):
1) You cannot do preemptive multitasking except by having a timer interrupt (okay, maybe one can also allow emulation).
2) Quantum key distribution (starting with BB84) depends crucially on the fact that the world is not classical.
But in general I agree with the article, it's more or less why I did not become a programmer.
You will indeed find the _magic_ to be no such thing when you dig deep down in the abstactions. Each part makes sense and is comprehensible.
However the magic is what emerges when the parts come together, the system becomes more than the sum of its parts.
Think of a car or a cellphone, an llm, it feels magical what it does, while the elementary parts do not consist of magic.
On the topic of how that implementation comptime in Go works: I've toyed with a similar idea for implementing constant folding/beta reduction — generate a temporary file with the relevant subset of definitions, insert the constant expression/function call you'd like to evaluate and... compile and run this file. It may not be the most performant thing to do but at least you get the correct semantics without writing a separate constexpr interpreter.
Sufficiently understood magic is by definition technology.
One of the cool things about being a kid is I didn't know how tf anything works. So I thought long and hard about how things work and just randomly guessing. When it turns out later there's a clear answer, it's very satisfying. But knowledge can also weigh down and discourage curiosity.
Charles Petzold's book "Code" was the book that revealed the magic behind computers for me. While I don't fully understand everything that happens with a computer, there's a confidence that I could if I needed to. It made tough problems feel solvable in a way I hadn't felt in other disciplines.
Going the other direction, towards higher levels of abstraction tends to strip away the magic too!
I recently write some robotics code using ROS, taking a step back I looked a the result and thought: Actually that is not much different conceptually from running K8s deployments coupled by Kafka.
In recent video about reverse game of life Alpha Phoenix hinted that sat solvers are magic: https://youtu.be/g8pjrVbdafY
Magic isn't real. But unnecessary magic can be annoying and sometimes doesn't pull it's weight.
Indeed. If it magically works, it can magically break.
interesting how a click bait title works just like magic and even more so gets people debating a so so premise in mystical terms
Birds aren't real
[dead]