Science Fiction

House of Leaves by Mark Z. Danielewski fits that model really well.

Player of Games is told in reverse, it works really well. Book of the New Sun is an Ouroborus, so you don't really understand it until you read it twice.

Edit: I mean "Use of Weapons" not Player of Games.

Redshirts by John Scalzi. A book about people who realize they’re characters in a badly-written TV show. Near the end, though, the main character starts to realize he’s the main character in a book about people who are characters in a TV show. Very surreal.

A lot of Douglas Coupland novels play with the form. Microserfs has a similar thing to 2312, except it is one of the characters random txt files on their desktop that lists words in a David Foster Wallace way.

Wallace also plays with the form of the book, particularly with end notes in Infinite Jest.

House of Leaves by Mark Z. Danielewski plays with the physical form of the book by rending two stories in parallel and playing all manner of interesting typesetting tricks.

Then there's Tristan Shandy by Laurence Sterne and published in 1759. It's argued that it is the first post-modern novel. The "author" is attempting to tell you the story of his life, but never really gets there because of endless digression.

You should check this page out: https://en.m.wikipedia.org/wiki/The_Life_and_Opinions_of_Tristram_Shandy,_Gentleman

Not sci-fi, but The Princess Bride.

You mean If on a Winter's Night a Traveler by Italo Calvino? Still do check it out

The Zombie Survival Guide by Max Brookes is a detailed fictional instruction manual. It's sequel is World War Z which is closer to a normal book, but still has an odd structure of creating a world through interviews ... and the The Zombie Survival Guide book exists within it.

Ilium/Olympos by Dan Simmons has a very interesting theory about reality and fiction, and what ties them together. Unfortunately, it's tied to a very late plot point that I would hate to spoil on a Lemmy comment.

Of course Flowers for Algernon by Daniel Keyes cannot be overlooked in terms of experimental writing.

There is a novel that will eternally be on my mind to wonder if it was drivel or I was not smart enough for it - The Troika by Stepan Chapman. It has some of the greatest imagination ever and you will probably not understand what is going on at all. Just find the description and decide if it's nonsense or worth it.

Not sci-fi, but Mister B Gone by Clive Barker was really good. The book is a conversation with a demon who is telling you his life story. It is the story of how he came to be trapped in the book you are reading.

I thought the book was super funny and surreal. I saw reviews after finishing that it is considered some of his worst work. That might be true, but the worst 24 carat bar of gold is still a 24 carat bar of gold. Well worth the read imo.

The Neverending Story

The Knife of Never Letting Go has a couple cool sections with fonts where as the action is rising you start turning pages faster.

Self-Reference ENGINE by Toh EnJoe definitely fits this description.

In the middle it feels like nonsensical chaos, by the end it all folds back in on itself.

Thursday Next series by Jasper FForde. Very meta, humorous.

"Building Stories" by Chris Ware. Not a book, but a deconstructed graphic novel that comes in a box full of different formats and sizes - from a newspaper to a tiny bubblegum sized strip. You read them in any order you please, learning stories of people who live in the same building, while also building your own path through the stories.

It's one of my most prized pieces of literature.

"S" by J.J. Abrams(yes, that one) and Doug Dorst

At face value it is a book about a man who became a member of a ship's crew and his journey. There is a second layer of editors notes left by the translator. Then there is a third layer of two previous readers of the book. There are puzzles and clues tucked into the pages of the book that you, as the third reader, try to understand what the fuck is actually going on.

I think Idaho Winter by Tony Burgess does this pretty well. It's a wild read.

Sophie's world. Also a nice introduction to philosophy

There's another book that I have heard of but neglected to write the name down, where the reader of the book is a character within the book, and the narrator speaks directly to you

If, on a winter's night, a traveller?

I think the most famous example is probably “Pale fire” by Nabokov. Not sci-fi, but very very fun! George Perec write some interesting concept books. One is about this apartment in Paris where every chapter just describes a random room in the building, but slowly tells a story of the inhabitants.

Stanislaw Lem's Imaginary Magnitude, a collection of introductions to non-existent books, is tangentially related to what you're asking here. Worth reading.

https://www.goodreads.com/en/book/show/953489

Cloud Atlas for its Russian nesting doll narrative structure.

Feersum Endjinn for how it plays with time, space, and even basic concepts of spelling and punctuation.

Generation X for its intwerweaving of advertising and informational sidebars into the text.

another older recommendation is jacques the fatalist, where jacques is attempting to tell the story of his lovelife but is constantly interrupted by other people telling their own stories, there is a "reader" who interrupts the narrator to ask questions, and the narrator at times gets bored of telling the story and asks the reader to fill in their own details. it also has an entire section directly from tristam shandy near the end.

not sci fi or fantasy but worth a read if you have the patience.

Thank you everyone for the suggestions! I didn't expect there would be so many!

I've put them all on my reading list and I'll get back to each of you when I've read your book.

(I don't know if it's possible to do a reply-all type thing and I don't want to type out the same comment 30 times so..)

Red X by David Demchuk is fiction mixed with autobiography.

Twenty-One Truths About Love by Matthew Dicks is a novel told entirely in list form.

I didn't care for either book, but they were definitely unique structures.

How High We Go in the Dark by Sequoia Nagamatsu was great and has been compared to Cloud Atlas, if that kind of structure is appealing to you.

It’s not sci-fi, but Hopscotch by Julio Cortazar lets you decide in what order to read the chapters.