AI Engineering @ the Speed of Startup: A modern core framework
The previous article in this series: AI Engineering @ the Speed of Startup: A model architecture
This article series explores a guideline for researchers and engineers to apply in AI research and development in a regulated setting in a sustainable fashion. It serves primarily as a thinking framework but greatly pragmatic.
We speak in Python, JavaScript, TypeScript and C#. We are always hiring — check out our careers page.
Contents
The industry-regulated startup constraintsAgile thinking into actionThe three layers of sustainable AI engineeringThe maintainability mindset (1st article)
├── Can we maintain this architecture?
├── What is the cost of change?
├── Systems design strategies, to begin with
├── A mini case study when microservices becomes a plague
└── A core monolithic system and satellite microservicesA model architecture (2nd article)
├── Laundry sorting workload types
├── General considerations for cloud-native AI solutions
├── Reuse, reuse, reuse
├── Designing for Compute-efficiency
├── Designing for Network and Storage-efficiency
└── A fine line between infrastructure and codeA modern core framework (this article)
├── Siloed research to production
└── Why does a consistent core framework work in the context of siloed research?
A modern core framework
Modern sounds cliche because we tend to think of ourselves as “modern” compared to our predecessors. The same mentality can be applied when we discover a new framework on the block over the prior. Both Django and Flask are remarkable frameworks for building APIs, have a massive following, an undeniable ecosystem, and their uses and advantages, but some may feel they are old. It would be best if you created your own modern core framework based on one of the popular such frameworks which would cut down on a lot of wheel reinvention.
We have stumbled upon FastAPI recently, and it feels modern. The creator expresses their humble opinion about what inspired them to create it. I like how they didn’t claim this is the best framework in the market and all other frameworks are poor performers. It feels like several popular libraries glued together and riding on the shoulders of the giants. The accompanying cookiecutter was quite surprising, with just enough of everything you need to deploy a production-grade API (as of March 2021, unfortunately, such level of enthusiasm diminishes over time for OSS). Here are the highlights:
- A Flask-like but cooperative multitasking based API framework
- Default security like JWT tokens
- Automatic Swagger UI
- Pydantic-based modeling, SQLAlchemy, Alembic, PostgreSQL, and PgAdmin plugged in
- RabbitMQ and Celery-based job queues with Flower
- Vue.js for dashboarding purposes
- Generates docker files for production-readiness
- Automatic SSL with renewals
It might sound like a Django-like boilerplate-driven development with all those features, but it is more like Flask. I do not sound convincing because I don’t want to. This section’s objective is to encourage you to consider modern frameworks and features commonly deployed nowadays, but the choice of a specific framework is up to you.
I have put together a monorepo solution template based on several patterns discussed in this article:
Siloed research to production
When we aim to achieve all of the above features to be deployed in production, multiple teams in silos in numerous various ways maybe reinventing the wheels and boilerplates over and over. The result is that we end up with an inconsistent and unintuitive setup that it becomes hard for anyone to pick up and be productive quickly. Staff mobility and onboarding become a daily hurdle. The amount of cash burned for repairing the damages, firefighting, debugging, and reconciliation of designs done in silos can hardly be justified. The economics of AI engineering’s sustainability deserves a different section, but this article is too small to cover that.
Some researchers prefer to work as a 1-(wo)man army and love to promote their PoCs to production? That’s great, but remind them that the end goal is to integrate it into the core framework. Their responsibilities include fully and actively supporting the integration, deployment, and customer service of their experiments that were turned into products.
Why does a consistent core framework work in the context of siloed research?
- Because it forces researchers and engineers to reuse, not reinvent.
- It forces them not to use a poorly designed boilerplate from the internet. They may/may not have put effort into understanding how good/bad it is for the job they are carrying out.
- It forces them to have skin in the game. If someone’s code breaks the build, it possibly went against a well-understood and running architecture, and there is no escaping it. i.e., they must fix it because they are blocking others.
- It challenges them to work under a specific set of operating conditions. Suppose you are running on Python 3 (or arbitrary packages, for that matter). If a team is comfortable with Python 2 and depends on a few AI libraries built for it, they may have to rethink their strategy. Always think modern. Outdated and unmaintained 3rd-party packages are a big challenge.
- It unifies the efforts in CI/CD and configuration management and makes it streamlined and consistent.
- It encourages and empowers independent decision-making. With a consistent base framework, scientists and engineers won’t have to wonder/ask around where their code should reside and how it should behave and interact.
Conclusion
This article series has explored a model for thinking about sustainable AI engineering's architectural foundation in a regulated setting that requires organizational adoption. Not all models fit everywhere. I’d be interested to learn how different your organization is in doing sustainable AI engineering.
Let us have a virtual coffee: tanzim.saqib@ezra.com.
