GitHub - navidseyedain/ALSTMobile: Real-time Android screen translation. Powered by Gemini AI for smart multimodal vision and Google ML Kit for full offline sup
📱 ALST — AI Live Screen Translation
[](https://github.com/navidseyedain/ALSTMobile#-alst--ai-live-screen-translation) **Real-time, in-place visual screen translation for Android powered by Google's Gemini 3.6 Flash & ML Kit.**
**Translate any game, comic, chat, or foreign UI in real-time.**
_Zero telemetry. No subscriptions. 100% Free & Open-Source._
- * *
📑 Table of Contents
[](https://github.com/navidseyedain/ALSTMobile#-table-of-contents)
- 🔬 How It Works (Architecture)
- 🚀 Quick Start & Installation
- 👤 Author
- * *
🌍 What is ALST?
[](https://github.com/navidseyedain/ALSTMobile#-what-is-alst)
**ALST (AI Live Screen Translation)** is an open-source Android utility that eliminates language barriers across all applications on your phone. Operating seamlessly via a non-intrusive floating overlay, ALST captures your active screen using hardware-accelerated `MediaProjection`, processes text using **Gemini 3.6 Flash's multimodal vision model**, and dynamically renders accurate translations directly over the original text coordinates using Android's `WindowManager`.
Whether you are playing untranslated foreign mobile games, reading raw Webtoons/Manga, chatting on messaging apps, or browsing international news, ALST provides an instant, contextual, and immersive reading experience without switching apps or taking manual screenshots.
> ### 💡 Primary Use Cases: > > [](https://github.com/navidseyedain/ALSTMobile#-primary-use-cases) > * 🎮 **Gaming**: Play Japanese/Korean RPGs and mobile titles with live dialog and menu overlays. > * 📖 **Comics & Manga**: Read webtoons, manhwa, and manga without waiting for scanlation groups. > * 💬 **Live Messaging**: Chat across WhatsApp, Telegram, or WeChat with instant auto-translation. > * 📰 **Global News & Research**: Read international apps and documents with full RTL/LTR script rendering.
- * *
✨ Key Features
[](https://github.com/navidseyedain/ALSTMobile#-key-features)
⚡ Single-Pass Multimodal Translation (Gemini 3.6 Flash)
[](https://github.com/navidseyedain/ALSTMobile#-single-pass-multimodal-translation-gemini-36-flash)
- **Zero OCR Bottlenecks**: Directly analyzes the raw screen frame buffer with Google's latest multimodal vision AI.
- **Context-Aware Semantic Understanding**: Understands idioms, slang, gaming terminology, and multi-line paragraph flows rather than translating words in isolation.
- **Auto Source Language Detection**: Simultaneously detects and translates mixed-language screens (e.g., Persian + English + Japanese) in a single API call.
- **Pixel-Accurate Coordinate Mapping**: Calculates normalized spatial bounding boxes to place translations exactly over the original text.
📴 Offline Resilience (Google ML Kit)
[](https://github.com/navidseyedain/ALSTMobile#-offline-resilience-google-ml-kit)
- Completely local, on-device OCR and neural translation for Latin-based languages when traveling or off-grid.
- Dynamic language model manager with live sync and download status.
🎯 Floating Action Button (FAB) & Instant Overlay
[](https://github.com/navidseyedain/ALSTMobile#-floating-action-button-fab--instant-overlay)
- **Draggable Frosted Glass FAB**: Move the trigger anywhere along your screen edges.
- **Quick Dismissal**: One-tap dismiss on any translation block, with auto-timeout safeguards (30s) to prevent screen obstruction.
- **Subtle Close Controller**: Long-press to reveal the dedicated mini close button to gracefully stop the background service.
- **Quick Settings Tile**: Start and stop the translation engine directly from Android's status bar / quick settings drop-down.
🔋 Battery & Memory Engineered
[](https://github.com/navidseyedain/ALSTMobile#-battery--memory-engineered)
- **Frame Throttling (300ms)**: Eliminates continuous GPU/CPU rendering cycles.
- **Memory Recycling**: Zero bitmap accumulation via reusable frame buffers and strict `ImageReader` lifecycle closures.
- **Android 14 Ready**: Fully compliant with `FOREGROUND_SERVICE_MEDIA_PROJECTION` requirements and graceful system UI disconnect callbacks.
- * *
📸 Screenshots & Visual Tour
[](https://github.com/navidseyedain/ALSTMobile#-screenshots--visual-tour)
🎨 Cyber-Glassmorphic Dashboard
[](https://github.com/navidseyedain/ALSTMobile#-cyber-glassmorphic-dashboard)
🌐 Real-Time In-Place Translation Showcase (4 Languages)
[](https://github.com/navidseyedain/ALSTMobile#-real-time-in-place-translation-showcase-4-languages) ALST translates raw screen text while preserving the layout, fonts, and spatial positioning:
⚡ Floating Controls & Quick Settings Integration
[](https://github.com/navidseyedain/ALSTMobile#-floating-controls--quick-settings-integration)
- * *
🔬 How It Works (Architecture)
[](https://github.com/navidseyedain/ALSTMobile#-how-it-works-architecture)Loading
flowchart TD A[User Taps Floating FAB] -->|Trigger Signal| B[ScreenTranslatorService] B -->|Request Frame| C[ScreenCaptureManager] C -->|MediaProjection + VirtualDisplay| D[ImageReader Frame Buffer] D -->|Throttle: 300ms| E{Active Engine Type}
%% Online Route E -->|Online Mode| F[Gemini 3.6 Flash API] F -->|Multimodal Vision Prompt| G[JSON Parser & Markdown Sanitizer] G -->|Extract Normalized Bounding Boxes + Text| H[OverlayManager]
%% Offline Route E -->|Offline Mode| I[ML Kit TextRecognition] I -->|Bounding Rects| J[ML Kit On-Device Translate] J -->|Translated Blocks| H
%% Render H -->|Calculate Screen Insets & Density| K[Jetpack Compose Overlay View] K -->|WindowManager.addView| L[Screen Translation Canvas Displayed] L -->|Tap Overlay / 30s Timeout| M[Clean Dismiss & Recycle Bitmaps]
- * *
📊 Feature Comparison
[](https://github.com/navidseyedain/ALSTMobile#-feature-comparison) | Feature | ALST (This Project) | Google Lens / Translate App | Typical Screen Translators | | --- | --- | --- | --- | | **Engine** | **Gemini 3.6 Flash + ML Kit** | Google Translate | Basic Tesseract OCR | | **Contextual Accuracy** | 🟢 **Ultra High (LLM-grade)** | 🟡 Medium | 🔴 Low (Literal) | | **Multimodal Single-Pass** | 🟢 **Yes (Vision AI)** | 🔴 No (Chained OCR+API) | 🔴 No | | **Mixed Language Screens** | 🟢 **Native Auto-Detect** | 🟡 Single Source Only | 🔴 Single Source Only | | **Overlay Placement** | 🟢 **In-place Spatial Matching** | 🔴 Full Screen Freeze/Card | 🟡 Floating Card | | **Privacy & Telemetry** | 🟢 **Zero Telemetry / BYOK** | 🔴 High Telemetry | 🔴 Heavy Ads & Trackers | | **Open Source** | 🟢 **100% Free & MIT Licensed** | 🔴 Proprietary | 🔴 Freemium / Paid |
- * *
🛠 Tech Stack & Dependencies
[](https://github.com/navidseyedain/ALSTMobile#-tech-stack--dependencies)
- **Core & Architecture**:
- **Language**: Kotlin 2.0+
- **UI System**: Jetpack Compose (Material 3)
- **Architecture**: Clean MVVM (Model-View-ViewModel) + Repository Pattern
- **Async / Reactive**: Kotlin Coroutines & Reactive `StateFlow` / `SharedFlow`
- **Dependency Injection**: Factory Pattern & Lifecycle-aware scopes
- **Android APIs**:
- **Screen Capture**: Android `MediaProjection`, `VirtualDisplay`, `ImageReader`
- **System UI**: `WindowManager` (`TYPE_APPLICATION_OVERLAY`), `TileService`
- **Storage**: `Jetpack DataStore (Preferences)`
- **AI & Machine Learning**:
- **Cloud LLM**: Google GenAI SDK (`gemini-3.6-flash`)
- **On-Device ML**: Google ML Kit (`text-recognition`, `translate`, `language-id`)
- * *
📂 Project Architecture
[](https://github.com/navidseyedain/ALSTMobile#-project-architecture)
``` ALSTMobile/ ├── app/ │ ├── src/main/ │ │ ├── java/com/alst/mobile/ │ │ │ ├── core/ │ │ │ │ ├── capture/ # MediaProjection & screen capture pipeline │ │ │ │ ├── ocr/ # ML Kit local text detection │ │ │ │ ├── overlay/ # WindowManager floating overlays & FAB │ │ │ │ └── translator/ # Gemini 3.6 Flash & ML Kit engine abstraction │ │ │ ├── data/ │ │ │ │ └── preferences/ # DataStore local settings repository │ │ │ ├── domain/ │ │ │ │ └── model/ # TranslationBlock, EngineType, Language models │ │ │ ├── service/ # Foreground translation & Quick Settings tile services │ │ │ └── ui/ │ │ │ ├── dashboard/ # Compose Glassmorphic dashboard & sheets │ │ │ └── theme/ # Cyber-dark theme, gradients, typography │ │ ├── res/ # Drawables, mipmaps, strings │ │ └── AndroidManifest.xml # Android 14 compliant permissions & services │ └── build.gradle.kts ├── docs/ # Screenshots, demo GIF, and media assets ├── gradle/ ├── README.md ├── README-fa.md └── LICENSE ```
- * *
🔒 Privacy & Security
[](https://github.com/navidseyedain/ALSTMobile#-privacy--security)
- **Bring Your Own Key (BYOK)**: ALST connects directly from your device to Google AI Studio's API endpoint. There is no middleman, proxy server, or custom backend.
- **Local Credential Storage**: Your Gemini API Key is stored securely on your device's private sandboxed DataStore storage.
- **No Analytics / Telemetry**: No tracking SDKs (Firebase Analytics, Crashlytics, Mixpanel, etc.) are included in the build.
- **Zero Retention**: Screen buffers are processed strictly in RAM and discarded immediately after rendering.
- * *
🚀 Quick Start & Installation
[](https://github.com/navidseyedain/ALSTMobile#-quick-start--installation)
📲 Option 1: Install Pre-built APK (Recommended)
[](https://github.com/navidseyedain/ALSTMobile#-option-1-install-pre-built-apk-recommended) 1. Download the latest release from Releases or click Direct APK Download (v1.0.0). 2. Install the `.apk` file on your Android device (ensure "Install from unknown sources" is allowed).
🛠 Option 2: Build from Source
[](https://github.com/navidseyedain/ALSTMobile#-option-2-build-from-source) #### Prerequisites
[](https://github.com/navidseyedain/ALSTMobile#prerequisites)
- **Android Studio** (Koala | Ladybug or newer recommended)
- **JDK**: Version 17
- **Device / Emulator**: Android 8.0 (API Level 26) up to Android 15 (API Level 35)
#### Build & Run
[](https://github.com/navidseyedain/ALSTMobile#build--run)
1. Clone the repository
git clone https://github.com/navidseyedain/ALSTMobile.git
2. Navigate to project root
cd ALSTMobile
3. Build debug APK using the included Gradle wrapper
./gradlew assembleDebug
Setup Gemini API Key
[](https://github.com/navidseyedain/ALSTMobile#setup-gemini-api-key) 1. Generate a free API key from Google AI Studio. 2. Launch the **ALST** app on your phone. 3. Paste the key in the **Gemini API Key** card and tap **Save**. 4. Grant **Overlay Permission** and tap the **Master Power Switch** to start translating!
- * *
🗺 Roadmap
[](https://github.com/navidseyedain/ALSTMobile#-roadmap)
- Multimodal single-pass translation via Gemini 3.6 Flash
- Offline local translation fallback via Google ML Kit
- Floating draggable FAB with gesture support & instant dismiss
- Glassmorphic Material 3 Dashboard with ambient blur
- Android 14 `MediaProjection` foreground service compliance
- Quick Settings Tile integration
- **Interactive Box Selection**: Drag to translate a specific rectangular screen region
- **Text-to-Speech (TTS)**: Listen to translated blocks aloud with native voices
- **History & Export**: Save translated snippets directly to markdown or clipboard
- * *
🤝 Contributing
[](https://github.com/navidseyedain/ALSTMobile#-contributing) Contributions are what make the open-source community an incredible place to learn, inspire, and create. Any contributions you make are **greatly appreciated**!
1. Fork the Project 2. Create your Feature Branch (`git checkout -b feature/AmazingFeature`) 3. Commit your Changes (`git commit -m 'feat: Add some AmazingFeature'`) 4. Push to the Branch (`git push origin feature/AmazingFeature`) 5. Open a Pull Request
- * *
📄 License
[](https://github.com/navidseyedain/ALSTMobile#-license) Distributed under the **MIT License**. See `LICENSE` for more information.
- * *
👤 Author
[](https://github.com/navidseyedain/ALSTMobile#-author) **Navid Seyedain**
- GitHub: @navidseyedain
- Projects: ALAD (AI Live Audio Dubbing) | ALST Mobile
Built with ❤️ using Kotlin & Jetpack Compose