Discovery Dive: Document Processing
Many PMs complain about a lack of discovery experience. User-centric work is a learnable skill. It's about "building habits".
I do discovery (almost) every week. This week I'm doing a rapid discovery on a field I don't know much about, namely systems that automate the handling of documents (using AI).
So come along to learn about how I discover (tactically) and what I've learned about this product segment.
You'll get a step-by-step walkthrough of how I discovered a problem space. And learn about Intelligent Document Processing along the way.
Let's dive into discovery mode.
Here's what's on the agenda for today
Introduction: WTH are Intelligent Document Processing Systems?
Market Research: Find user problems.
Discovery Calls: Qualitative understanding.
Experimentation: Prototype-based validation of one idea.
Conclusion: Key learnings, challenges, next steps.
Sources: All references I used.
Introduction
WTH are Intelligent Document Processing Systems?
What the hell is an Intelligent Document Processing System or IDPS? Let's start with that.
Simply speaking IDPS are tools that allow companies to automate document processing. Think about getting invoice PDFs via mail and automatically transferring data into the internal accounting system. That saves the billing & accounting department loads of time.
That's the idea. Pretty useful, right?
Never really thought about that space before. So I wanted to find out more after stumbling over one particularly neat-looking solution: rossum.ai.
Tactical Tip 1: First thing I do when jumping into discovery is write down every keyword I don't know. Or key concepts I don't think I know enough about. I rank them in order of my knowledge or familiarity with the keyword/concept.
Making a little plan
In this case:
- IDPS
- Compliance
- Integrations
- OCR
So IDPS is a new acronym for me. I intuitively understand what's meant by it. But I should not rely on that.
Same with compliance. Documents, processed by a third party, sensitive information ... I can imagine the legal nightmare that is. But still, want to dig deeper there.
Integrations are pretty clear, still want to learn about specific challenges here.
I've worked with OCR before, briefly.
Cool, now I have a few specifics to do my research on. This is by no means a fixed discovery plan. I have to stay nimble when I know so little.
What don't I know?
Next, I brainstorm 5 mins on what my Known Unknowns are. I'm using that to kickstart my market research.
Later on, differentiating between Biases and Knowledge will become critical. For now, it isn't. I focus on what I don't know.
Market Research
First, let's simply look for existing software solutions in the space.
I'm looking for authentic user voices and user problems. No marketing blog posts.
I go to Reddit, Quora and the AppStore.
So, there seems to be no 'document processing' related Subreddit. Therefore I go broader. Try to find an OCR Subreddit. Find out that it means "Obstacle Course Racing". Which is entertaining but not what I was looking for.
So I go broader. r/MachineLearning. 2 + Million members. Search for "IDPS" there.
No success. First learning, IDPS seems not to be a popular topic among average people.
Assumption 1: Not many non-enterprise use cases.
I go broader. OCR, there has to be something about OCR!
And there is.
Gold. I've found a real user (username: younggamech) with a real problem. It also seems like a pretty common problem. Let's look at the comments.
What am I learning here?
By reading nested comments I can get a feel for how users speak about their problems and solutions. After being given https://rossum.ai/ as a solution the user immediately asks for API capabilities.
Careful. I have just one user here. N=1. The goal is not to derive the 'priorities' of anything here. The goal is to even know which questions to ask.
I take down two questions for my impromptu interview guide.
- How do you process documents right now?
- What's the role of APIs in your document workflow?
(if no) Have you thought about how to use APIs in your document workflow?
(if yes) Walk me through that API workflow, please.
With that let's keep moving. In the spirit of keeping this a single blog post, I won't dive into the details of each Reddit post I find here.
You now know the drill. I keep looking through forums etc., I collect screenshots of key findings on a Miro board and add assumptions and interview questions to my lists.
Tactical Tip 2: Learn to find the right info on Reddit and Quora. First search for a specific Subreddit on your topic. If that doesn't exist, go broader and search within that Subreddit. Same applies to Quora. AppStore Reviews are a great source of data as well.
Discovery Calls
Being armed with some market understanding I jot down a loose interview guide. I'm still in discovery mode, therefore I'm trying to stay flexible and go with the flow in interviews.
Interviewed two people in very different roles. Here come the shortened transcripts of those user interviews.
The goal was to understand their current workflow with digital documents and how they process them.
Have they thought about using an IDPS?
Do their user problems match the ones an IDPS would solve?
Let's find out.
First Discovery Call:
P.D. - Technical Consultant at B2B Tech Company
Context on the potential user: Working with big event organisers, such as fares and conferences. Contracts make up the major share of document handling. Tickets for the events are already sold online, so everything including invoicing is fully automated already.
How do you process documents right now? Could be pdfs, print-outs or any other form.
Two document flows. Contracts with customers at the beginning of each project. And then, additional contacts with third parties (service providers).
In which format did you receive and send out these documents?
Per email (PDF). But for the service providers, we often printed them out. We worked with the documents online to edit and sign them (Salesforce and Docusign) when possible. But for service providers, we didn't have such an online process, so we received them by mail, printed them out, signed them if everything was ok, and scanned them again to send them back.
What's the role of APIs in your document workflow?
No.
(if no) Have you thought about how to use APIs in your document workflow?
No. But would be amazing to have an automated process to reduce the back and forth. But I didn't think further about it.
Second Discovery Call:
Project Lead for Energy and Sustainability Consulting for Real Estate Developers
Context on the potential user: Working with real estate developers with 10k-100k apartments on how to plan out energy savings and achieve carbon neutrality. Large amounts of unstructured data from differing sources.
How do you process documents right now? Could be pdfs, print-outs or any other form.
90 % digital, per mail, as Excel or PDF. I send the clients' template excels and they fill them out with their data.
Clients don't always fill them correctly.
We get PDFs for energy passports of real estate, but they are not always clear to which real estate object they belong.
If so, you need to open each energy passport and try to read (several hundred) out a few data fields each. Copy them into our Excel.
This work is done by interns and regular coworkers manually if clients don't do it themselves.
(if no) Have you thought about how to use APIs in your document workflow?
Yes, asked for a tool already. But ways of working are quite established. We also don't need it that often.
How's that?
Partly you can somewhat copy out of PDF into Excel and hope that the rows match up. Sometimes you can convert from PDF to Word and then to Excel.
In short: the pain is not big enough. Not every client has that problem and it's not a massive part of the overall work.
An example: Google earth pro takes in CSV data that comes from the customers. Namely, the addresses we get from clients. We get that as Excel.
We enforce clients to give us at least address data in a predictable format.
Assumption 2: Internal friction to adopt a new tool is higher than perceived value.
Assumption 3: Potential users don't think of IDPS as generic solutions. They believe their problems are unique.
Experimentation
Now we have some market understanding, we have some user insights. Next, I'll gather my assumptions, prioritise these by risk factor and do a prototype-based validation of that hypothesis.
Assumption 1: Not many non-enterprise use cases.
Assumption 2: Internal friction to adopt a new tool is higher than perceived value.
Assumption 3: Potential users don't think of IDPS as generic solutions. They believe their problems are unique.
Okay, I came up with these assumptions by doing more than the two discovery calls - but this post is supposed to be an example. And it is already getting loooong. Who's still with me? 👀
I don't care about non-enterprise use cases right now. Enterprise use cases are great - B2B is a great market. So Assumption 1, I discard.
I feel pretty confident in Assumption 3. Only technically savvy people think about generalised solutions, while domain experts think about their specific task (they'd like to automate). Still, something to experiment with.
Assumption 2 is a critical one for me. Is the adoption of a new tool like rossum.ai hindered by its value proposition (meaning it's not really needed) or by its adoption mechanism (the whole enterprise needs to adopt it)?
Getting to Experimentation
Now, I'm moving fast and scrappy here. I'm skipping some steps but the intention is to show the whole cycle from 0 knowledge to validated learnings. So let's at least look at the most critical assumption in more detail.
The critical assumption or hypothesis:
Internal friction to adopt a new tool is higher than perceived value.
The experiment to gain data on this assumption:
Make a self-serve, no-lock-in free solution to extract data from PDFs without creating an account. Look at smallpdf as an example.
The method:
Cheapest to get quantitative data would be a fake-door test explaining the solution. Here's an example landing page design I mocked up.
Conclusion
Next Steps
So, what are the next steps:
We deploy this on a landing page site like carrd.co. We add analytics to the page and the choose files button. And we see how our conversion looks.
We can use the same mock-up to show to potential users. This way we get qualitative data on how such an offering would be perceived. What concerns do end-users have (e.g. data privacy)?
I'm shortening the discovery cycle here, as I worked on this timeboxed for a set number of hours. But the idea was to show you how I'd approach a new space and walk you through step-by-step what I do to gain insights.
Challenges
The main challenges I foresee in this space are
- the large range of use cases to cover and understand
- the data privacy concerns hindering fast onboarding
- identifying the purchase-decision-maker vs. the end-user and serving their need respectively
Key learnings
I've learned a lot. Coming from only a gut feeling about such a niche concept as IDPS to some actual understanding.
- Internal processes are well-engrained, and IDP-Systems need to interface with these processes or replace them. Challenging for convincing new clients, as the adoption barrier might be quite high.
- Identifying the buyer persona vs. the user persona is going to be a core challenge. Finding ways to product-enabled sales here is going to require a deep understanding of organisations.
- I believe product-led growth opportunities exist by providing low-friction, free tooling.
Much more discovery to do. Much more to learn.
I hope you could learn a little bit by looking over my shoulder for this.
Sources
https://www.reddit.com/r/MachineLearning/comments/wwbumm/d_ocr_models_for_invoice_reading/
Haissam Abdul Malak