You’ve got an idea. Maybe you noticed some unused data sitting in your log files. Maybe you were inspired by some tech you saw on ai.google. Maybe your competitors have been busy and you need to upgrade your product to stay competitive. So where do you start?
AI budget
You’ll need a budget. Some AI tools, data, and libraries are free, some cost a little money, some cost a lot of money. The people who build these tools and libraries into products for you will definitely cost money. They will probably be working on your project full-time, with no time for other duties or software maintenance.
Idea, specified
The computer system can’t tell you what you ought to build. You need to be able to describe exactly what it is that you want the software to do. In fact, you need to be capable of doing the task by hand. If you want to build a spellchecker, create a small sample dataset and go through the process by hand of exactly what the computer will do. The computer will read a document, and then… how does it know if a word is misspelled? Oops, you need a dictionary to compare each word to. Ok, you get a dictionary, one of the words in your document isn’t in the dictionary because… oops, it’s a company name. So you will need to decide how you want the system to deal with that. Another word in your document isn’t in the dictionary because it’s misspelled; does your spellchecker automatically correct it? Does your spellchecker suggest alternatives to the user? Etc. “Take a dataset of emails and cluster the emails to find topics” is too vague to do by hand, and the computer can’t do it either, until you are more specific about what you want.
AI expertise
As you are planning this project, you should discuss your tentative plans and ideas with an AI expert. The expert will provide valuable guidance on project feasibility and ML capabilities, as well as suggesting alternative solutions that will reduce costs and improve the final product quality.
Product expertise
Many decisions about building AI software are not technical decisions, but are actually product decisions. “Should we extract company names or signatory names first?” and “Should we summarize the article text or just give the users a list of titles?” are product decisions. Technically, either can be done, but the controlling factor is matching the actual product to the product vision and serving it on-time in-budget to the relevant users or investors.
Data
If you’re building a solution with mostly 3rd party libraries/models/SAAS, you’ll need a moderate amount of data. If you’re building a solution with custom-built ML models, you’ll need a large amount of data. If you’re not sure which you’re building, ask your AI Expert. Even a 3rd-party library implementation will react uniquely to different types of data, so you need a moderate amount of data (perhaps 200-700 news articles, or 10,000 tweets; it totally depends on your particular system) to test your system and simulate real-life use. Custom-built models might require 3000 news articles for 5-way topic classification, or 400,000 tweets for named entity extraction, etc.
Time
There are hundreds of pre-built AI solutions out there. There are millions of AI problems out there. If your AI problem is one of the lucky hundreds, congrats! You can get your system running and tested within a few weeks. Otherwise, it’s very likely your project will take months or years to build and test. Your task must be clarified, your data must be hand-analyzed, related solutions must be researched in the literature, a working prototype must be built, users must test it, the bulk of errors must be identified and corrected, the solution must be implemented into your production code, your AI engineering team must be on standby to catch and correct new types of errors as they come through the system, such as from changing dependency libraries, changing input data formats, changing data topic distributions, etc.
Engineers
AI software is built by software engineers. There are different kinds of software engineers. Some specialize in certain types of databases and their architecture and maintenance. Some are very fluent in a particular programming language (such as C++, Java, or Python) and will write whatever you need in that language. Some are experts in the cluster of technologies (such as HTML, CSS, React, Javascript) necessary to build a good website. Some are generalists and have a little bit of experience with everything. Some have experience with datasets and statistics and probability distributions and related tools (R, MySQL, Jupyter Notebook). Some have experience with formatting, storing, and transporting data. As you plan out your project with your AI expert and Product expert, your AI expert will be able to tell you which types of software engineers are needed for your team, and assist you in recruiting them.
That’s the overall picture. You're already ahead of the curve!
