Useful-Move5612

Hi,

Having developed multiple AI based products now and running them in production I run into problems again and again and wonder whether others have a solution for this.

I have multiple prompts that are expect to return a JSON with a given structure. Even now since you can hand the output type to openAI I see no difference in result quality. One has to be very careful in prompt constructing and model choice. A single word differently placed can lead to broken JSON output. Same with switching models eg from 3.5 Turbo preview to 4 with otherwise same inputs can reduce output quality.

And that’s just one issue. With volume I sometimes see random hallucinations in language. Multi language handling is tricky in general.

That lead me to think what’s needed in this early stage of AI building is a rigid testing environment. More than just a playground but rather an environment that can mass test prompts and results on an ongoing basis and monitor results, suggest improvements and so on. Just to bring what’s needed to seriously build business products rather than play use cases for single user Environments.

Does anyone of you know of such tool out there? Anyone else having theses problems with their commercial products?

Hey 👋

After lerking for the past years I wanted to share my thoughts and experiences on building an AI app after just releasing one myself.

TLDR: AI is no moat and building with early AI is scary.

As a lot of you I also was excited using the new tech to build something awesome and since I needed to refresh my Spanish vocabulary and always struggled to find the right app to do so, I spent 300 hours building the one I always wanted myself.

Background : I learned Spanish in school 15 years ago, now the concepts but am missing the vocabulary specific on topics (e.g. on buying stuff in the tool store). Language learn apps are too many whistles for me and it’s hard to narrow down a good point of entry for my level and as a developer I am too lazy to use Flashcard apps like quizlet or anki to write the vocabulary by hand on the phone.

That’s why I built KardsAI which lets you generate flashcards from any pdf document, pasted text or via prompt on any topic. That way I can now just type in „most useful Spanish vocabulary for buying tools“ and got a exactly what I needed. Awesome right?

So why do I think it might have still been an error to build it myself? Well because if you check the big flashcard apps like quizlet today you will notice that they integrated AI themselves to include something in that direction. 6 months ago they did not.

Though their solution lacks the, for me essential, prompt part and is much more limited in the size of document, those are factors they could adapt quite easily. Sure other features are a bit different as well but the main USP differentiation is now missing.

In the end it will be very hard to compete against an giant app with 10 million downloads and a pretty competitive market environment.

On the plus side I have now the solution to my personal problem and learned a ton about building an AI based app. As a side-note here: it’s scary for scaled systems. The amount of things to consider to get it to work consistently is pretty astounding. Model steering, fine tuning prompts, temperature, p values, trying to force a consistent json structure, multi language handling, fallback calls and asynchronous handling for the model overloads during certain hours are all things that make it scary to use as an application foundation in my eyes.

Looking forward to your feedback. Did you have similar experiences? Ah and does anyone by chance have tips on good tools or platforms to create ad sets, especially in video form? Think I know the usual ones but had hoped by now some helpful new ones would have popped up.

Lastly if you are looking for a new flashcard app, maybe consider KardsAI and help me grab a .0001% market share and get some feedback 😅

Timo