| Abstract | The rapid expansion of consumer IoT devices has increased the need for scalable, automated testing solutions. Manual methods are often slow, error-prone, and inadequate for capturing real-world IoT complexities. Existing frameworks typically lack comprehensiveness, quantifiable metrics, and support for cascading failure scenarios. This paper introduces Thoth, a lightweight, end-to-end IoT testing framework that addresses these limitations. Thoth enables holistic evaluation through integrated support for performance, reliability, recovery, security, and load testing. It also incorporates standardized metrics and real-time failure simulations, including cascading faults. We evaluated Thoth using eight test cases in a real-world health-monitoring setup involving a smartwatch, edge gateway, and cloud infrastructure. Key metrics—such as fault detection time, recovery speed, data loss, and energy usage—were logged and analyzed. Results show that Thoth detects faults in as little as 2.5 seconds, recovers in under 1 second, limits data loss to a few points, and maintains sub-1% energy overhead. These findings highlight its effectiveness for low-intrusion testing in resource-constrained environments. By combining scenario-driven design with reproducible, metrics-based evaluation, Thoth fills key gaps in IoT testing. |
|---|
