Initial Situation
GRS wanted to bring certified gemstones to life digitally via its own app for iOS and Android, one that would run reliably even on future operating system versions. At the same time, the company migrated its Laboratory Information Management System (LIMS) to a new solution. The idea behind it: The app identifies a gemstone using a machine learning (ML) model in the backend, retrieves the corresponding video data via API, and displays it directly in augmented reality over the stone. This transforms a certified gemstone into an interactive experience.
Challenge
The challenge lay in automatic visual recognition. Gemstones offer a camera almost nothing to latch onto:
- few distinctive features
- many reflections
- constantly changing light
Standard frameworks like ARKit andARCorewere quickly overwhelmed by this. GRS needed a recognition system that could reliably identify the correct stone even under these conditions.
Approach
soxes collaborated with GRS to develop a proprietary machine learning model for image analysis, specifically trained on the characteristics of certified gemstones. This model was integrated into a native app that connects to the new LIMS via standardized APIs and automatically retrieves all relevant data. This ensures that recognition, data integration, and AR visualization work together seamlessly.
Solution
The GRS AR app combines artificial intelligence with modern app technology: it precisely identifies certified gemstones and superimposes supplementary content stably over the stone in augmented reality. The specially trained ML model ensures high recognition accuracy and reliable media placement, even when viewing angles or lighting conditions change. The native app for iOS and Android is intuitive to use, directly connected to the new LIMS, and designed to be flexibly expanded and rolled out internationally.
Technologies Used
- Apps: natively developed with Swift (iOS) and Kotlin (Android)
- AR and recognition: ARKit and ARCore, supplemented by a custom-trained machine learning model for precise image recognition and positioning
- Backend: Python stack with Django REST Framework and MySQL
- API documentation: Swagger
- Versioning, testing, and releases: Bitbucket, Azure Pipelines, Postman, Google Play Console, and TestFlight
