Acquire necessary data (scraping, processing into useful format storing somewhere accessible i.e. cloud storage or local hard drive) python, requests, pandas, google cloud storage.
Prepare training dataset (feature engineering, usually process into csv) python pandas, sometimes random external libraries sometimes in other languages.
Design initial model architecture, get training to actually work usually involves a lot of debugging. All in jupyter notebook using pytorch, local if possible else on gcp
Evaluate model, loss + accuracy good to confirm model is learning and sanity check but mostly this involves using the eye test in a real world scenario so build poc product integration at this stage (usually python web api and some frontend for interacting either python desktop app or js web app just running locally)
Iterate 3 + 4 until model is good enough for prod.
Implement prod deployment usually python web api often get frontend specialist involved at this point to handle any potential frontend integrations.
Maintain prod integration as bugs come up and new features are requested. Also continously eval model performance (depends on domain).
Come back a year later and work through the whole process again adding incremental improvements to each step.
That's the usual life cycle of a project for me personally. Could take anywhere from 1 week to 3 months to get to working prod implementation depending on the project. Any longer than that and need to seriously reevaluate the approach.
Basically just regular software engineer dutues plus ml stuff.
That's the usual life cycle of a project for me personally. Could take anywhere from 1 week to 3 months to get to working prod implementation depending on the project. Any longer than that and need to seriously reevaluate the approach.
Basically just regular software engineer dutues plus ml stuff.