Development of a personalized optimization method for evaluating business ideas using LLMs

▤ Summary

AIST Solutions and Stockmark developed a method that personalizes LLM-based evaluation of business ideas. The research shows that averaging expert judgments can produce scores that do not match any real evaluator, while personalized judges better reflect individual expert criteria. The team built the PBIG-DATA benchmark from 300 product ideas and about 3,000 expert evaluations to test the approach. The findings support using personalized AI judges in new business screening, R&D topic discovery, and idea evaluation workflows. The research will be presented at ACL 2026 and already feeds into Stockmark’s business-planning AI agent.

▦ Linked Companies

Stockmark

$1M to $5M