Socket map for check virtual users and virtual domains

[Aravinda-HWK]

📌 Description

This PR adds a socketmap service for Postfix to validate virtual users and domains against Thunder IDP, eliminating the need for local user and domain tables in the shared.db database. The service implements the Postfix socketmap protocol with netstring encoding for efficient TCP-based lookups.

• Closes [FEATURE] Create Policy decision service for check the user existance #280

---

🔍 Changes Made

• New socketmap service implementing Postfix TCP table lookup protocol
• Netstring protocol implementation for safe binary data transmission
• Thunder IDP integration for real-time user and domain validation
• Multi-level caching system with configurable TTL to reduce API calls
• Token refresh mechanism with automatic background renewal
• Security hardening:
  • Input validation to prevent filter injection attacks
  • Path traversal protection for domain lookups
  • Memory exhaustion protection (10MB netstring limit)
  • Proper escaping of user input in API queries
• Organizational Unit (OU) mapping from domain hierarchy (e.g., dev.silver.openmail.lk → openmail.lk/silver/dev)
• Connection management with timeouts and graceful shutdown
• Comprehensive logging with visual indicators for debugging

---

✅ Checklist (Email System)

• Core services tested (SMTP, IMAP, mail storage, end-to-end delivery)
• Security & compliance verified (auth via Thunder IDP, TLS, DKIM/SPF/DMARC, spam/virus filtering)
• Configuration & deployment checked (configs generated, Docker/Compose updated)
• Reliability confirmed (error handling, logging, monitoring)
• Documentation & usage notes updated (README, deployment, API)

---

🧪 Testing Instructions

1. Start the socketmap service:

   cd services/socketmap
   go run main.go

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a new, dedicated socket map service to handle virtual user and domain existence checks for the Postfix SMTP server. This change significantly enhances the flexibility and dynamism of user validation by decoupling it from the shared.db SQLite database, allowing for easier integration with external identity providers or user management systems.

Highlights

• New Socketmap Service Introduced: A new Go-based socketmap-server has been added to the Docker Compose stack. This service is responsible for dynamically validating the existence of virtual users and domains, replacing the previous SQLite-based lookup mechanism.
• Postfix Configuration Updated: The gen-postfix-conf.sh script has been modified to configure Postfix's virtual_mailbox_maps to utilize the new socketmap-server for user validation, ensuring that Postfix queries the dedicated service for recipient checks.
• Netstring Protocol Implemented: The socketmap service now correctly implements the Postfix-specific netstring protocol for communication. This crucial fix resolves issues where Postfix was connecting but not sending data, ensuring proper interaction between Postfix and the new service.
• Comprehensive Documentation and Tools: Extensive documentation, including architecture diagrams, Docker deployment guides, quick start instructions, and a detailed troubleshooting guide, has been added. Diagnostic and interactive testing scripts are also included to aid in setup and debugging.

Changelog

• SOCKETMAP_CHANGES.md
  • Documented the key changes and deployment steps for the socketmap integration.
• services/config-scripts/gen-postfix-conf.sh
  • Updated Postfix configuration generation to use the new socketmap service for virtual_mailbox_maps.
  • Clarified comments regarding the use of SQLite for domains/aliases and socketmap for user validation.
• services/docker-compose.yaml
  • Added the socketmap-server service definition to the Docker Compose configuration.
  • Established a dependency from smtp-server to socketmap-server.
• services/socketmap/.gitignore
  • Configured Git to ignore Go build artifacts, test binaries, dependency directories, and common IDE files for the new service.
• services/socketmap/ARCHITECTURE.md
  • Provided detailed diagrams illustrating the mail flow, Docker network topology, configuration flow, and data flow with the new socketmap service.
  • Included a comparison of before (SQLite) vs. after (Socketmap) architectures, caching details, and protocol specifics.
• services/socketmap/DOCKER_DEPLOYMENT.md
  • Documented the Docker deployment process, service definition, Postfix integration, and production considerations for the socketmap service.
  • Included troubleshooting steps and environment variable explanations.
• services/socketmap/Dockerfile
  • Defined the multi-stage Docker build process for the Go socketmap application, using golang:1.21-alpine for building and alpine:latest for the final image.
• services/socketmap/INTEGRATION_SUMMARY.md
  • Summarized the integration of the socketmap service, detailing modified and new files, architecture changes, and deployment instructions.
  • Outlined benefits, migration paths, monitoring, and security considerations.
• services/socketmap/LOGGING_UPDATE.md
  • Described significant enhancements made to the socketmap service's logging for better debugging and troubleshooting, including startup, connection, data reception, request processing, cache, and error details.
• services/socketmap/NETSTRING_FIX.md
  • Explained the resolution of a critical issue by implementing the correct netstring protocol for Postfix socketmap communication.
  • Detailed the netstring encoding/decoding functions and updated connection handling.
• services/socketmap/QUICKSTART.md
  • Provided a quick reference guide for starting, testing, and configuring the socketmap service, including manual testing and Postfix configuration examples.
• services/socketmap/README.md
  • Documented the socketmap service's purpose, deployment options, protocol details, test users, logging, and customization points.
  • Included production considerations and troubleshooting tips.
• services/socketmap/TROUBLESHOOTING.md
  • Offered a comprehensive guide for diagnosing and resolving common issues related to Postfix not sending data to the socketmap service, including configuration checks, direct testing, and log analysis.
• services/socketmap/diagnostic.sh
  • Added a shell script to automate diagnostic checks for socketmap connectivity and configuration, providing detailed output for troubleshooting.
• services/socketmap/go.mod
  • Initialized the Go module for the socketmap service with Go version 1.21.
• services/socketmap/interactive-test.sh
  • Provided an interactive shell script for manual testing of email addresses against the socketmap service.
• services/socketmap/main.go
  • Implemented the core Go socketmap service, including netstring protocol handling for reading requests and writing responses.
  • Added environment variable parsing for host and port configuration.
  • Integrated a simple in-memory cache for user existence checks.
  • Defined test user validation logic, accepting specific users and users from allowed domains.
• services/socketmap/test-netstring.sh
  • Added a specific test script to verify the netstring protocol implementation of the socketmap service, including encoding and decoding functions.
• services/socketmap/test.sh
  • Provided a general automated test suite for the socketmap service's functionality, covering valid/invalid users, cache behavior, and invalid requests.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a new socketmap-server Go service to handle virtual user and domain lookups for Postfix, replacing the previous SQLite-based approach. The changes include the Go service itself, Docker integration via docker-compose.yaml, a Dockerfile for building the service, and extensive documentation and test scripts. While this is a significant and valuable feature, the review has identified several critical and high-severity issues. There is a critical race condition in the in-memory cache implementation in main.go that needs to be addressed with a mutex. Additionally, there are widespread inconsistencies between the implemented netstring protocol in the Go service and the plain-text protocol used in most of the test scripts and documentation, rendering them incorrect and non-functional. Other suggestions include improving the Dockerfile for security and reproducibility, and adding a healthcheck to the service in docker-compose.yaml.

[CLAassistant]

All committers have signed the CLA.

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[Aravinda-HWK]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces a new socketmap-server service written in Go to handle Postfix virtual domain and user lookups, replacing the previous SQLite-based mechanism. While the architectural change is sound and the Go code is clean, several critical security vulnerabilities were identified. These include hardcoded administrative credentials, potential Denial of Service via memory exhaustion, and multiple injection vulnerabilities (Filter Injection and Path Traversal) due to insufficient validation of untrusted input. Additionally, the socketmap-server container runs with excessive privileges (as root with the Docker socket mounted), and TLS verification is disabled for communication with the Thunder IDP, further increasing the attack surface. There is also a critical bug in the token refresh logic that will cause authentication to fail after the initial token expires. Addressing these issues is essential before deployment.

[maneeshaxyz]

LGTM.