Recently, when I looked at the discussions about Ask-Data on social software, I found that many of the comments came from personal experience. I thought it might be interesting to collect them and make an inductive analysis.
So we collected 472 comments, including 267 main comments, and compiled them into this article.
The collected keywords cover: Ask-Data, chatBI, text2sql, data analysis agent, data querying implemented. Exclude advertising posts.
From the perspective of the sample, negative and questioning comments account for a clear majority.
What is everyone mainly arguing about?
On the surface, everyone is discussing "Does Ask-Data work?" But after merging the comments, there are five types of questions that really appear repeatedly:
1. Whether the data foundation supports
This is the category that appears most frequently and has the strongest consensus.
Typical comments:
There are two huge bottlenecks in chatbi. One is that the data base is not good, and garbage in garbage out...If data governance is not done well, it will definitely not work if you use it directly.If the data foundation is not well prepared, it may be difficult to achieve results.
The meaning of such comments is clear:
If the underlying data is inherently messy, Ask-Data will not turn the messy data into good answers, it will just output errors faster.
2. Are the semantic layer, indicator system, and caliber definition mature?
Many comments pushed the question a step further: Even if the original data is available, it does not mean that the question is correct.
Typical comments:
The most critical ones are data governance and semantic modelingMost of them are stuck in indexing. Currently, AIdata querying can only be based on the index system.The data that the leader wants is all imaginary. If you ask the person in charge of the business to come, it will take him a long time to know what caliber of data the leader wants.
The core here is not whether SQL can be generated, but:
- Are the indicators clearly defined?
- Is the business caliber unified?
- Whether the problem can be expressed in a structured way
If these problems are not solved first, the stronger the model, the more wrong it will look like right.
3. Can the cost of accuracy and trust be affordable?
The consensus among these types of comments is also very strong.
Typical comments:
The crisis of trust caused by incorrect data makes it impossible to promote Chatbi.The leader's expectations are the most troublesome. The MVP version is the first to be 95% accurate.How much does it cost to implement 80% to 95%?
The biggest difference between Ask-Data and ordinary AI assistants is:
Once it goes wrong, users will hardly regard it as an "inspiration tool", but will directly regard it as "untrustworthy".
In other words, the accuracy requirement for this type of product is not "just usable", but close to "no obvious mistakes". To go from 80% to 95%, it is often not a model adjustment, but a lot of governance, modeling, rules, verification and organizational collaboration costs.
4. Are users’ real needs “free to ask questions”?
There is a very important point of disagreement in this criticism: many people do not think that "free data querying" is a real need.
Typical comments:
This thing itself has a bit of pseudo-demand, and is more from the perspective of technology research and development.I think data access is not a problem for most companies. The problem is the data infrastructureMost of the time I just need a few numbers. Over designed
These users believe that customers may need more:
- Fast data access under defined calibers
- High-confidence answers in fixed scenarios
- Kanban explanations, exception explanations, and business suggestions
If a product bets on "open natural language analysis" from the start, it's easy to overestimate the true need.
5. Is ROI and organizational promotion cost-effective?
In fact, many comments are no longer concerned about technology, but are asking:
In order to accomplish this, is the organizational cost worth it?
Typical comments:
Many companies are doing it, but they haven’t done it well. This is just a gimmick for the time being.It's just a purely technology-driven project...that's a ppt project, a political achievement projectA very important job of technology sales is to reduce customers’ unrealistic expectations about AI
In other words, many projects die not from model effects, but from:
expectations too high
Scene definition is too broad
Business departments are not cooperative
Landing cycle is too long
The delivery cost is much higher than the perceived value
Why do people generally find it “difficult to land”?
If the above discussion is condensed into a more essential sentence:
Ask-Data is difficult to implement, not because “the model is not smart enough”, but because it happens to fall at an intersection that requires high basic capabilities of the enterprise.
This intersection also requires:
- Data layer is available
- Clear indicator layer
- business language interpretable
- Users know what they are asking
- The system can provide credible answers
- Enterprises are willing to bear the costs of governance and iteration
And the reality is, most businesses don’t do all six of these things at the same time.
That’s why a judgment appears repeatedly in the comments:
It's not that data querying has no value, but that most companies don't have the prerequisites to do it well.
This is why many practitioners redefine the problem as:
- The real difficulty is not
text2sql - What is really difficult is to organize the company’s data and business expressions so that they can be stably consumed
Which objections hold the most weight?
Not all naysayers are equally valuable. In this batch of data, the most significant objections mainly come from three types of people:
1. People who have done implementation or project delivery
Such comments usually don't stop at "I don't think it's useful", but will clearly point out the stuck points:
- Data base
- Indicator system
- real-time
- complex issues
- Cost vs. accuracy trade-off
Such comments are highly credible because they are about landing resistance, not emotional judgments.
2. People who understand data analysis and BI systems
Such people often point out a key contradiction:
Things that data analysis did poorly in the past will not automatically become better just because AI is added.
The importance of this type of perspective is that it prevents teams from misdiagnosing “analysis problems” as “model problems.”
3. People who are sensitive to business needs
Such comments mainly express:
- Users don’t necessarily really want to “ask questions freely”
- Many demands are essentially fixed business analysis
- Many bosses can’t explain their caliber
The value of this kind of feedback is that it can help the product move from "what technology can do" back to "what users really need."
Relatively speaking, although simple comments such as "AI is not good" and "it's all gimmicks" have emotional signals, they do not constitute a basis for decision-making when viewed alone.
Does it mean "Ask-Data has no value"
Discussion does not mean "Ask-Data has no value".
A more accurate statement is:
The value of "general-purpose, open-ended Ask-Data that replaces analysts' imagination" is overestimated; but "narrow-scenario, strong-constraint, semi-automatic, and verifiable" Ask-Data is still valuable.
This type of value usually appears in the following scenarios:
1. Enterprises that already have a relatively mature indicator system
If the enterprise has completed good data governance, unified indicator caliber, and semantic layer construction, then Ask-Data can be used as:
- A more natural query entrance
- A portal for managers to obtain business data on their own
- A Kanban explanation and auxiliary analysis portal
2. Scenarios of fixed problems, fixed caliber, and fixed data sources
For example:
- Sales Daily Q&A
- Inventory exception description
- Review of operational activities
- Questioning financial operating indicators
What these scenarios have in common is:
- Problem scope is controllable
- The result has a calibration basis
- Users are more tolerant
3. Human-machine collaboration rather than pure automatic replacement
A more realistic form is not "complete self-service analysis of business", but:
- AI helps understand problems
- AI helps route to the right indicator or service
- AI helps interpret results
- Key conclusions are retained for manual verification or review
This is easier to implement than "directly replacing analysts" and is more in line with the real feedback in the comments.
Based on these data, more worthy directions
Based on the discussed data, four categories of valuable directions were sorted out.
Direction 1: Change from "free data querying" to "scenario-based business analysis assistant"
Instead of asking users to ask any questions, we provide them around a few high-frequency business scenarios:
- Standard question template
- Structured questioning
- Indicator explanation
- Exception attribution suggestions
This type of direction is more in line with the real needs in the comments and is easier to build trust.
Direction 2: Change from "SQL generation" to "semantic layer and indicator governance enhancement"
This criticism has repeatedly proven:
What many companies really lack is not a model that is better at writing SQL, but:
- Indicator definition
- Metadata annotation
- business semantics mapping
- Data lineage and caliber management
Therefore, rather than making a "smarter entrance to data querying", it may be more valuable to make a "front-end layer that enables data querying".
Direction 3: Change from “Analyst Replacement” to “Analysis Improvement Tool”
The more realistic value is not necessarily to be used directly by the business, but to help:
- data analyst
- BI team
- Business analysis team
To finish faster:
- Check the numbers
- Correct caliber
- Find relevant indicators
- Generate draft explanation
- form a standard reply
This type of scenario has a different tolerance for accuracy, and it is easier to get early positive feedback.
Direction 4: Turn capabilities into consulting/delivery/MVP verification services instead of building large and complete products first
Judging from these comments, the real difficulty is highly dependent on the internal situation of the company. This means that a general platform is not currently the optimal entry point.
A more realistic way of cutting it might be:
- First make a narrow scenario verification service
- First, help customers sort out indicators and semantic layers
- First make verifiable results on one or two business issues
- Then decide which capabilities are worthy of productization
This is also consistent with the positioning of this project itself: first build an opportunity verification workbench, rather than first build a full-featured platform.
Conclusion
If I were to condense these 472 comments into one sentence that could really guide my next steps, I would write it like this:
Ask-Data is not a false proposition, but "open universal data querying" is most likely an over-promised proposition.
For someone who wants to continue making products based on this type of opportunity, the most important inspiration from these comments is:
- Don’t make it a universal portal that “can ask you anything”
- It should be made into an analysis assistant that “can stably give credible answers in specific scenarios”
If you further condense it into a sentence that is more suitable for product decision-making:
What is really worth doing is not a "universal Ask-Data platform", but an "analysis assistant with semantic layer and rule constraints around specific business scenarios."
This is closer to the real needs behind this criticism and closer to the opportunities that can be implemented.
More Ask-Data related technologies:
Ask-Data technical route and selection
Unified semantic model solution for enterprise agent