In the previous article, we explored how technology transformed market research and gave birth to 'Research Platforms'. We also looked into the benefits of these platforms and identified some of the shortcomings when they are used as DIY Research Platforms. This article is a story of Thomas, a tale that resonates with many researchers and decision makers who ventured into the DIY approach, with added insights on AI, Data Science, and Generative AI.
Thomas is a diligent researcher at a leading e-commerce company, one growing rapidly with ambitions to lead its category. Ambitions come with opportunities and challenges, and some of these challenges were transferred to Thomas. One of the biggest was to increase the number of research initiatives to provide the brand with accurate information on market needs and competitors. Overwhelmed by the number of projects, limited resources, and rising costs, Thomas decided to invest in a DIY Research Platform enhanced with AI and data science capabilities. This platform promised to streamline research processes, leveraging AI for more efficient data analysis and Generative AI for automated report generation, at a fraction of the cost of hiring an agency. After thorough demos and comparisons, Thomas made the purchase.
Stage 1: The Learning Phase
Following the procurement of the platform, the first task for Thomas was to train his team. A two-month training plan was devised. The team was enthusiastic, aiming to become self sufficient for all research needs. The platform supplier explained various aspects, demonstrated features, and created multiple dummy projects to illustrate the platform's functionalities.
After a week of training, Thomas attempted to recreate an old project on the new platform. He managed only 10% of the task and felt lost with the rest. The skip and branching logics, enhanced with AI driven insights, seemed too complex. Despite multiple attempts, he gave up. His team faced similar challenges and requested another training session. Here is what they learned:
- Some features required additional licensing costs.
- List building, branching logic, and minimum read, despite AI assistance, remained complex.
- Some logics were not possible on the platform as they were deemed unnecessary by most users.
- For complex survey logics, the vendor offered programming services at a nominal cost.
Stage 2: The Working Phase
After the two-month training period, Thomas realized the team still needed agency partners for most projects, using the platform only for basic tasks. For almost 4-5 months, a hybrid model was adopted: some projects were done internally, some with the DIY Platform provider's help, and others outsourced. This worked until senior management questioned the continued use of agencies.
Thomas explained to senior management:
- Survey programming, data processing, and dashboard creation, even with AI, were complex and required programming skills beyond the team's expertise.
- Some projects were so complex that only seasoned programmers could handle them, justifying the support from the platform vendor.
- The team's workload necessitated outsourcing some projects to trusted partners.
- The company worked on advanced surveys requiring agency support.
Stage 3: The Hiring Phase
Uncomfortable with the DIY Platform's underutilization, Thomas proposed hiring skilled resources to fully leverage the platform. The business case was made to support hiring, recruitment efforts were launched, and eventually, the right people were onboarded. The new team members quickly started contributing to projects.
Stage 4: The Renewal Phase
With the new team of eight programmers and an equal number of researchers, most projects were handled internally, with only 10-15% outsourced. As project volume increased, the platform's resources were exhausted, necessitating license renewal. Meetings were held, numbers crunched, and the business case for renewal was presented:
- Per project cost when outsourced: X
- Per project cost using the DIY Platform: 1.8X (including license, salaries, recruitment, and training costs)
- Although the team could deliver more projects with the DIY platform, the overall cost increased due to additional factors.
Not all DIY research platforms are equal. Some are sophisticated, leveraging AI and Generative AI to do more than what an average researcher needs, while others are only suitable for basic projects. DIY tools or platforms are the future for smart researchers, but robust evaluation is necessary to avoid pitfalls. Before choosing a DIY Research Platform, understand your research's complexity, align it with platform features, and evaluate the proportion of work it can handle. Ensure discussions on licensing and recurring costs support your future interests. AI and data science can significantly enhance these platforms, but they also require proper expertise and resources to fully realize their potential.