Why I was Brought on

My role was to design a use-case Machine Learning Operations (MLOps) tool from scratch. But after working on this goal for a year, focusing entirely on technical users, we saw that Konan, at least the front-end part, was barely being used inside organizations.

Data scientists used our API to perform most of their goals and business users didn't understand our frontend from all the technical jargon aimed at data scientist.

Finding Our North Star

Having a sticky and highly visible product was essential to the survival of the company. This lead to a pivotal change in our focus. We decided to go back to our original vision of a use-case MLOps tool with a heavy emphasis on ,use-case.

We looked at our most used use case, credit scoring, as the perfect place to start. This caused a whole deep dive into understanding what credit scoring is being used for, who uses it, how do they use it, what other tools are used in its place, and so much more.

Now, my role was to turn Konan from a MLOps tool, focused on engineers and data scientists, into a Use-case MLOPS tool focused on risk officers and business users.

First: Who are risk officers anyway?

To answer this question, we hired a risk officer for 3 reasons;

1. Deeply understand risk officer needs and barriers.
2. Tap into his network of risk officers
3. Refine our credit scoring models.

This helped us jump start our research by setting up 4+ interviews with risk officers across banks and business that provide loans/installments.

Shocking Truth: AI is Not Important (Right Now)

Maybe, it's not so shocking after all. It makes sense that unless the core issues of their workflows are not being addressed, AI seemed far away. This forced us to focus more on quality of life features. Here are the issues we identified;

1. Data Visibility: Getting visibility using excel sheets, badly taken screenshots, badly designed presentations and emails, were tedious and time consuming.
2. Policy Testing: Testing policies was time-consuming and required a ton of manual copy-pasting through excel sheets
3. Automate Policy Decisions: AI is cool, but our customers are focused more on automating policy decisions using a combination of their own policy rules and regulation

Restructuring Konan

We had two structural problems;

1. Workflow outgrew its size and didn't quite fit inside a project with a model. A user can create a project; only use it's MLOPS features, or only use its workflow features. This seemed confusing and restrictive for any future additions. We needed a way to separate different types of projects.
2. It was common that users created duplicate projects to start from a clean slate. This seemed like a hack. We needed a way to allow testing more freely without worrying about muddying the data

Better Decisions Through Simplicity

With all the change of focus and additional knowledge we gained; we wanted to simplify and design the most essential Konan experience that aligns with our vision. Here are the main ideas that stuck out;

• Separate domains: Models & Workflows
• Separate stages: Test, Share, Deploy, Monitor, & Repeat
• Models Marketplace (Future)
• Common Templates (Future)

Cultural Change

This was one of the most challenging pivots I've experienced, but we did it. Changing our focus from MLOPS into Workflows with MLOPS on the side, really challenged the team, as most of us joined the company to work solely on an AI product. But as with any product, things change. This change was great for the company, and our customers.

↑ Top