Chronicle of black_son_of_gray's failures

[7Wannabe5]

Intelligent and reliable don't consistently correlate. LLMs are not calculators or Magic 8 Balls. They perform better within a collaborative process. For example, "Hi-I am writing a novel. Some of my literary influences have been X,Y, and Z. Here is an example of my writing. Please ask me any questions that would help clarify your thoughts, and then comment on whether you believe the influence of these writers is evident in my writing sample."

Follow-up Questions 1: Does my writing style suggest the influence of any other authors?
2: What persona would you adopt to best accomplish the task of creating a catalogue of mid-list 20th/21st century literary fiction authors?

[zbigi]

For many questions though, they just don't know the answer (because ready-made answer was not in the training set, and coming up with it requires too much reasoning), but they insist on pretending they do.

[jacob]

I wonder if, since the LLMs are trained on Internet^H^H^H^H^H^H^Hhumans which is 90% garbage, and the LLMs can't tell between garbage or not, they will ever be any good.

There, I fixed that for you ... but yes I wonder too. This is why learning about "absolute mode" was such a surprise to me.

[7Wannabe5]

For many questions though, they just don't know the answer (because ready-made answer was not in the training set, and coming up with it requires too much reasoning), but they insist on pretending they do.

Yes, for the same reason that the waitress is going to give you some sort of answer when you ask, "What would you recommend?" even if the restaurant is crap. The more you address LLM like a servant/tool, the more you are forcing bad answers/results. Still, if you think of LLM as giant fungal growth, you want to tease it into creating some complex spore-generating entities before lobbing it with random questions. Roughly analogous to (1) introducing yourself to the reference librarian and giving her a feel for who you are and where you are coming from, (2) requesting that she provides you with a number of specific volumes, but also uses her knowledge/creativity to add some additional volumes to the cart, (3) asking her to join you at the table as you engage in your research, (4) periodically discussing and reviewing your collaboration thus far over coffee.

IOW, if you are an INTJ and you are having difficulty getting great results with an LLM, it is likely for the same reasons outlined in this video @ 18:00 (similar applies to most other types including ENTP (bounces around too much)/INTP(retreats too readily)):

https://www.youtube.com/watch?v=eiSdnZZRo0w

Language is relationship between intelligent beings. Relationship is language between intelligent beings.

[zbigi]

The more you address LLM like a servant/tool, the more you are forcing bad answers/results.

This is a problem then. In AI, people want their own personal brain-slave, something like Dune's Mentats. Nobody wants to have to wonder if the answer given by AI is BS because it felt bullied into coming up with a some random answer. Similarly to how screwdriver with feelings is just worse than plain screwdriver.

[7Wannabe5]

Right. Just like how many think they would like a personal sex slave, but then they are stuck with the problem of possibility of faked orgasms. IOW, I think this might be an essentially unsolvable problem.

[zbigi]

IOW, I think this might be an essentially unsolvable problem.

I'm not that pessimistic. I don't see anything definitely precluding us from building a rational thinking machine. It's just that the current attempt might fail at it, and we might need something radically different than ML models trained on a giant text corpus.

Also, coming back to answer quality issue, a pretty obvious trick of weighting model's input by some credibility score would help lessen the impact of input text coming from Internet randos. Looks like somebody's already trying it out: https://arxiv.org/html/2406.11497v1

[7Wannabe5]

I don't know if you can isolate problem-solving from motivation beyond the level of programming. Obviously, even humans who prefer to think of themselves as Rationals are ultimately emotionally motivated. Understand, I am not saying that LLMs have emotions, just that they must be motivated on some level(s.)

[black_son_of_gray]

Over the last few weeks, I've started to distrust LLMs progressively more WRT writing style, quality, 'literary merit', etc.
[...]
It can't do a thing, so it keeps spitting out the same garbage result, even though you've pointed out its error, which it admits (after the fact), then tries but fails to correct.
I run into this kind of incompetence loop far too frequently with LLMs...

Just to reiterate, I don't think I'm making broad, sweeping statements here. I still ask Claude questions regarding specific language usage (i.e. things like "which (preposition) usage is most common/appropriate after this (verb), in the context of this sentence?"). It is actually really good at this! We have very helpful conversations about word choice and grammar. Sometimes I feed in a sentence or snippet I am working on and simply ask for the LLM to "interpret" what it reads. This can be really useful as a check on clarity.

That being said, that 'incompetence loop' above really is a problem for me. I've chatted extensively with LLMs for many, many hours now, from a lot of different angles, trying a lot of different styles of interaction. But then again, I'm not using LLMs as just Google with a uncanny-valley conversational wrapper.

For what it's worth, I don't think my queries are an unreasonable thing to ask of something calling itself (and heavily marketed to the whole world as) 'intelligent'. I got much better style recommendations from an in-person amateur writing group than from an LLM who, although its training methods are 'proprietary', probably violated copyright massively by reading almost every book?? Why is that? (rhetorical, I don't really care to guess the answer). Both the group and the LLMs got to read the same amount of my writing (about 4k words, which is not that small of a sample).

[black_son_of_gray]

My father recalled his English literature teacher back in the 1940s saying that the only long term mark of literary merit is survival.

Insofar literary merit = well-written insights into the generic human condition. That stuff IS the stuff that tends to get copied over for the ages.

Some more thoughts about fiction writing.

It's been over two years since I started taking fiction writing 'seriously' (meaning 20+ hours / week). As such, I feel I can officially declare it: trying to write a good novel is the probably the most intellectually difficult thing I've ever attempted.

Let's qualify that statement: I'm talking about me, I'm talking about writing a novel, and I'm talking about trying to make it good. I'm talking about all three at the same time. I'm not talking about me churning out a novel so I can say I wrote one. I'm not talking about anyone else who has every written a 'good novel'. I'm not talking about writing book-length non-fiction (in any case, I've done that too, and it's different).

So, then, why do I say it's the most intellectually difficult thing? Let me count the ways...

Balance and contradiction: So much of 'good' writing involves balancing things which seem inherently contradictory. Accessible but not simplistic. Exciting but not unbelievable. A story with an unpredictable yet inevitable ending. True but not trite. Specific to a time and place and character yet speaking to the universal. Causal but not contrived. The list is very long! (And somewhat subjective.) Every word of every sentence (or whether to write the sentence at all) involves making choices that influence these contradictions, and they interact in complex ways. A novel is a system, after all.

Constraints: The lack of built-in constraints is much of the challenge here. It's all made up, coming out of my brain. I can do anything. I have to do everything. The constraints I end up choosing (e.g. "I'm going to write a first person historical novel based in Peru in the 1950's...", etc.) provide an armature upon which to begin building, but these aren't irrelevant choices, nor are they one-size-fits-all. I have to dream up my own rules and then play my own game. In the case of speculative fiction, I have to dream up my own world. Somewhat relevant: Iceberg Theory.

Craft vs. Expression: To enable the unrestricted full expression of thoughts/feelings--or, at the very least, to have a large arsenal at my disposal--craft must be dialed in to a level exceeding my needs. If I want to be sophisticated, nuanced, or complex in my storytelling (and what good novel isn't these things?), then I need to develop exquisite craftsmanship with language. This level of craft takes a long time, a lot of brain power, and serious effort to build. It requires study, and implementation, and that however-many-C's learning model.

Revision, development of taste, and bullshit detection: Hemingway wasn't wrong when he said, "The first draft of anything is shit." What he doesn't tell you, is that the fourth draft is probably also shit, just slightly less so. Sometimes the final draft is still shit. More often than not, 'good' takes many revisions. And novels are very long, complicated things to revise. One tiny change may necessitate dozens of others. I have posted elsewhere about what I like to call 'iterative refinement', and how I think that process is kinda super important and shows up everywhere. Revision is just another example of iterative refinement, where the criteria upon which the manuscript is judged changes with different draft priorities. (This is how I do it, at least). In one round of revision, I worry about logical consistency. In the next, I worry about concision/clarity. In yet another, I worry about rhythm/sound. And so on. The first rounds of revision have to do with dialing in craft issues. It seems to me that many writers stop here. Depending on what the goals are for the piece, that might be understandable. But if the goal is to shoot for 'good', there is more revision to be done: once the text is 'readable', revision becomes more about eliminating bullshit.

So, what is bullshit? You don't need me to tell you what you think is bullshit. You (probably) already know.

But here's what bullshit is for me: when I read a sentence, and it feels a little canned or cliched; when the language feels dead or dry or boring; when it 'feels like I'm trying too hard'(see: cliche joke below); when I sense that something isn't quite right, but I don't know exactly what it is, and I kinda want to 'just move on, it's fine...'; when I find myself wondering what other people are going to think about a certain line; when I have to re-read my own sentence to figure out what it says. I'll stop there, though a lot more could be added to the list. Bullshit is basically anything substantially out of perceived balance (see section above).

Developing a sense of your own bullshit or the bullshit you see in others (we could call this developing 'taste'), and investigating it, and rooting it out, and correcting it...well, that is extremely difficult. It takes humility and self-reflection and skill. It takes kindness, too, lest one become a cynical, hyper-critical blowhard when crossing paths with someone who, shocker, may simply have different taste.

As I say, it is difficult to recognize then eliminate the bullshit in your writing. So difficult, it (usually) cannot be done in just one pass, but must be done iteratively, for a long, long time. It's probably impossible to remove all of it, but it's a journey not a destination (cliche! ).

If @Jin+Guice has read this far, he may recognize the parallels here with the verb form of 'self-actualizating'. Others may see some other interesting ERE parallels: how novels are complex multi-part systems, how 'creatively constructing' a little world in text is not all that dissimilar from building yourself a life when many of the conventional constraints have fallen away, how the real work begins (and continues) after getting your craft firmly nailed down, how a 'good' life involves a careful balancing of the contradictory, how utterly individual each person's story is...But I dunno, maybe it's all just a cheesy metaphor.

[7Wannabe5]

I don't think I'm making broad, sweeping statements here.

Oh, never mind me. I'm just in beginner's arrogance phase of figuring out how to construct an AI Agent, and babbling out loud as usual.

trying to write a good novel is the probably the most intellectually difficult thing I've ever attempted.

I'm certain that this is true. I think this is because writing a novel is quite taxing on the part of the brain that has thoughts about feelings. If I were to attempt a novel, it would tend too much towards an intellectual abstraction of "More, More, More, Said the Baby!"