SMB AI Power User - Competitive Evaluations

As an SMB AI Power User, you’ll help evaluate how AI assistants perform against the real-world needs of small and medium-sized businesses. You’ll use your own business experience and daily workflows as a practical testing environment, comparing competing AI tools side by side. Your work will cover marketing, customer communication, research, pricing, scheduling, content creation, and other everyday business tasks. You’ll identify where AI assistants succeed, where they fall short, and which limitations could affect paying business users. The role combines hands-on business operations, structured testing, and detailed technical reporting. It’s an independent, output-focused opportunity for someone who already relies heavily on generative AI in their professional life. Accountabilities

• Run head-to-head prompts and multi-turn workflows across AI assistants, using realistic tasks drawn from small-business operations and marketing.
• Build and maintain a library of practical SMB test cases covering advertising copy, content planning, customer communications, competitor research, pricing, scheduling, bookkeeping, and related workflows.
• Test AI capabilities across social media and commerce environments as well as standalone conversational interfaces.
• Identify and document failure modes, including inaccurate responses, unnecessary refusals, poor instruction-following, tone issues, brand-unsafe content, formatting problems, lost context, and unusable image or video outputs.
• Produce clear, reproducible evaluation reports detailing the prompts used, AI model and interface, expected results, actual results, severity, and business impact.
• Compare the value delivered by bundled AI subscriptions with competing paid AI assistants based on realistic business use cases and comparable costs.
• Evaluate AI tools from the perspective of a real paying business user, focusing on practical usefulness rather than purely technical performance.
• Maintain a consistent weekly delivery cadence while working independently against defined evaluation criteria.

• Current or recent experience as the owner, founder, sole proprietor, or day-to-day operator of a small or medium-sized business, ideally with approximately 1-200 employees.
• Daily, habitual use of generative AI tools for both business operations and marketing; occasional or experimental AI use is not sufficient.
• Hands-on management of a business presence on Instagram and Facebook, including posts, Stories, Reels, comments, and customer messages.
• Practical experience creating, launching, and optimizing paid advertising campaigns on Meta platforms using real advertising budgets.
• Based in the United States and authorized to work there, according to the original role requirements.
• Availability to complete the agreed deliverables and workload, potentially up to 40 hours per week.
• Exceptional written English, with the ability to explain AI failures clearly and precisely enough for an engineer to reproduce them without additional clarification.
• Strong analytical and observational skills, with an ability to distinguish meaningful product limitations from minor inconsistencies.
• Comfortable working independently, following structured evaluation criteria, documenting findings, and consistently meeting weekly output expectations.
• Experience with paid AI subscriptions such as ChatGPT Plus/Pro, Claude Pro/Max, Gemini Advanced, or similar is a plus.
• Experience using multiple AI assistants simultaneously and comparing their practical strengths and limitations is advantageous.
• Familiarity with AI automations, agents, APIs, no-code integrations, custom GPTs, or similar tools is a plus.
• Previous experience in model evaluation, red teaming, data annotation, quality assurance, or structured user research is beneficial.
• Familiarity with SMB tools such as Shopify, Square, QuickBooks, Klaviyo, Canva, CapCut, or Later is an advantage.

• Contractor compensation of $40-$50 per hour, with the exact rate determined based on experience, expertise, and geographic location.
• Remote working arrangement.
• Opportunity to work directly with cutting-edge generative AI products and competitive AI assistants.
• Hands-on opportunity to influence how AI products serve real-world small-business users.
• Work centered on practical business scenarios rather than purely theoretical AI evaluation.
• Flexible contractor engagement with the potential for up to 40 hours per week depending on the agreed scope and timeline.
• Opportunity to contribute structured insights that can inform product quality, usability, and competitive positioning.
• As an independent contractor, you provide your own secure computer and high-speed internet.
• Company-sponsored benefits such as health insurance and paid time off are not included with the contractor engagement., Deze baan Landelijk gemiddeld IT ICT vacatures gemiddeld België gemiddeld