The Onboarding
Ghost:
Engineering a Java
Email Bot from Nothing

When I joined Fixme, I quickly discovered a ghost in our machine. It wasn't a bug, a performance issue, or a missing feature. It was an absence. A complete void where a critical business process should have been. I'm talking about our customer onboarding. When a new user, brimming with excitement and curiosity, signed up for our platform, they were met with absolute, deafening silence.

There was no welcome email. No "getting started" guide. No confirmation of their registration. It was as if they had shouted into a canyon and heard no echo. For the small businesses and vendors who spent considerable time meticulously building their profiles on our marketplace, this silence was even more damaging. They were left in a state of limbo, completely unaware if their submission was received, under review, approved, or simply lost to the digital ether.

This wasn't a minor flaw in the user experience; it was a fundamental business failure. We were investing heavily in marketing and acquisition to bring people through the front door, only to have them walk into an empty, unlit room. Their initial spark of intent was being extinguished by our own inaction.

But Fixme had no pre-existing system, no templates, and no tools for this. I realized then that I couldn't just write a proposal. I had to become the solution — the strategist who defined the user journey, the designer who crafted the first impression, and the engineer who built the machine to make it all real.

Phase 1: Architecting an Experience from a Void

Before a single line of code could be written, I had to be a product manager and a UX designer. I began by diagramming the entire user journey from the first ad click to the final signup. The diagram starkly visualized the "onboarding black hole."

With the problem visualized, I designed a two-pronged strategic response tailored to our two primary user groups.

A plain-text, system-generated email would feel cold and undermine the entire effort. I used the design tool Deepsite to meticulously craft two distinct, fully-responsive HTML email templates. Every color, font, and image was chosen to align with our brand identity and create a powerful, welcoming first impression.

The finished templates are live at:

• francisj2nd-user-welcome-email.static.hf.space — Customer welcome guide
• francisj2nd-business-approval-email.static.hf.space — Vendor launch kit

Now, finally, the mission was clear. I had designed a world-class onboarding experience from nothing. All that was missing was the technology to deliver it automatically, reliably, and instantly.

Phase 2: The First Attempt — A Necessary, Fragile Prototype

To build momentum and prove the concept to my team, I needed a rapid proof-of-concept. Waiting for a full-scale engineering project to be greenlit could take months. I chose Node.js and Puppeteer — JavaScript's ubiquity and Puppeteer's high-level browser control API made it perfect for a working prototype fast.

// Core snippet from the early Node.js prototype
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://fixme.ng/xyz');
  await page.type('input[name="email"]', 'my-admin-email@gmail.com');
  await page.type('input[name="password"]', 'my-secret-password');
  await page.click('button[type="submit"]');
  await page.waitForNavigation({ waitUntil: 'networkidle0' });
  await page.goto('https://manager.fixme.ng/users');
  await page.waitForSelector('table tbody tr');

  const usersOnPage = await page.evaluate(() => {
    const rows = Array.from(document.querySelectorAll('table tbody tr'));
    return rows.map(row => {
      const cells = row.querySelectorAll('td');
      return {
        firstName: cells[2]?.innerText.trim(),
        email:     cells[11]?.innerText.trim(),
      };
    });
  });
  await browser.close();
})();

The prototype worked. But as I ran it more frequently, I started living the developer's nightmare. The reliance on highly specific CSS selectors (td:nth-child(10)) was its Achilles' heel. A front-end developer on another team adding a single column in the admin UI would silently break my entire data-scraping logic.

The whatsapp-web.js library I'd explored for outreach was even worse — a constant cat-and-mouse game with WhatsApp's engineers, and a clear Terms of Service violation that posed a serious business risk.

The final straw: a user had signed up with the email address blessed@com. My email library rightfully rejected it as invalid, threw an exception, and because my error handling was a single, global try-catch block, the entire process died. Hundreds of valid users that came after that single bad entry were left in the onboarding void.

The prototype was an immense success as a learning tool. It proved the value of the project and exposed every engineering challenge that a professional-grade solution would need to overcome. It was time to graduate. It was time to build it right, with Java.

Phase 3: The Re-Platforming — Engineering for a Professional Java Ecosystem

The decision to migrate to Java was a commitment to robustness, maintainability, and alignment with Fixme's core infrastructure. For this undertaking, I leveraged Google AI Studio as a powerful co-pilot — my pair-programmer for boilerplate and logic translation, letting a one-man team move with the speed of a larger one.

From the very first file I structured the application with a clear separation of concerns. Nine distinct classes, each with a single well-defined responsibility:

Phase 4: Engineering a Bulletproof Scraper and a Resilient System

This phase was about methodically solving each of the problems that had plagued the prototype.

Dual-Target Scraping: Users and Businesses

BotRunner orchestrates two independent scraping runs per execution — one against the users table, one against the businesses table. Each table has a different column layout, so a dedicated column map is defined for each target:

// Column indices (0-based) for each table type
private static final Map<String, Integer> USER_COLUMN_MAP = Map.of(
    "firstName", 2, "lastName", 3, "phone", 10, "email", 11, "signUpDate", 13
);
private static final Map<String, Integer> BUSINESS_COLUMN_MAP = Map.of(
    "firstName", 2, "lastName", 3, "phone", 9, "email", 10, "signUpDate", 12
);

This abstraction means the same scrapeTable() method handles both targets cleanly. Each new business owner gets the celebratory "Your Business is Now Live" email; each new customer gets the visual quick-start welcome guide.

Solving Pagination with Loop-Back Detection

The fatal flaw of the prototype was its inability to reliably determine when it had reached the last page. A naïve approach — counting pages or waiting a fixed time — is fragile. My solution is more robust: the bot collects the phone numbers from every row on the current page into a Set. After clicking "Next," it collects the phone numbers from the new page. If the two sets are identical, the admin panel has looped back to page one, which means the bot has seen everything.

// Core pagination logic from scrapeTable()
Set<String> currentPhones = extractPhones(usersOnPage);
Locator nextButton = page.locator("a:has-text('Next')");
if (!nextButton.isVisible()) break;
String cssClass = nextButton.getAttribute("class");
if (cssClass != null && cssClass.contains("disabled")) break;
nextButton.click();
page.waitForTimeout(1500);
List<ScrapedUser> nextPageUsers = extractRowsFromTable(page, columnMap);
Set<String> nextPhones = extractPhones(nextPageUsers);
if (nextPhones.equals(currentPhones)) {
    System.out.println("ℹ️ Phone numbers unchanged — reached last page.");
    break;
}
usersOnPage = nextPageUsers;

This approach is fundamentally more reliable than timing heuristics because it validates the actual data content, not the browser's animation state.

Two-Stage Email Validation

The blessed@com incident made it clear that a single regex check is not enough. The production EmailValidationService runs every email through two independent gates before any send attempt is made:

1. Syntax check: A strict regex validates the structure of the local part and domain, including TLD length (2–7 characters).
2. DNS MX record lookup: Even if the syntax is valid, the domain must have a real Mail Exchange record. A domain like example.fake will pass most regex checks but cannot actually receive email.

// From EmailValidationService.java
public boolean isEmailValid(String email) {
    if (!isSyntacticallyValid(email)) {
        System.out.println("⚠️ Invalid syntax for: " + email);
        return false;
    }
    if (!domainHasMxRecords(email)) {
        System.out.println("⚠️ No MX records found for domain in: " + email);
        return false;
    }
    return true;
}

private boolean domainHasMxRecords(String email) {
    try {
        String domain = email.substring(email.indexOf('@') + 1);
        Hashtable<String, String> env = new Hashtable<>();
        env.put("java.naming.factory.initial", "com.sun.jndi.dns.DnsContextFactory");
        DirContext ctx = new InitialDirContext(env);
        Attributes attrs = ctx.getAttributes(domain, new String[]{"MX"});
        Attribute mx = attrs.get("MX");
        return mx != null && mx.size() > 0;
    } catch (Exception e) {
        return false; // Any failure means the domain can't receive email
    }
}

Dual SMTP with Intelligent State-Based Failover

A single mail server is a single point of failure. MailConfig provisions two completely independent JavaMailSender beans — primary on port 587 (STARTTLS) and fallback on port 465 (SSL/TLS). EmailService maintains a stateful PreferredSender enum that remembers which sender succeeded last. Network-level failures trigger automatic retry on the other sender; application-level rejections (invalid recipient) are not retried.

// From EmailService.java — the connection-issue discriminator
private boolean isConnectionIssue(MailException ex) {
    Throwable cause = ex.getRootCause();
    if (cause instanceof ConnectException
            || cause instanceof MailConnectException) {
        return true;
    }
    String msg = ex.getMessage() != null ? ex.getMessage() : "";
    return msg.contains("Got bad greeting") || msg.contains("[EOF]");
}

A mail server outage on port 587 will never silently drop emails. The bot detects it on the first failure, switches to port 465, and continues the batch without interruption.

File-Based Memory and Deduplication

StorageService maintains two persistent JSON files: users.json and businesses.json. On startup, previously processed records are loaded. After scraping, only records whose phone numbers are absent from the stored set are processed. On completion, new records are merged into the master list using phone number as the deduplication key.

// From StorageService.java
public void writeRecords(File file, List<ScrapedUser> oldRecords,
                        List<ScrapedUser> newRecords) {
    Map<String, ScrapedUser> merged = new LinkedHashMap<>();
    // Old records first, then new records overwrite on collision
    for (ScrapedUser u : oldRecords) merged.put(u.getPhone(), u);
    for (ScrapedUser u : newRecords) merged.put(u.getPhone(), u);
    try (FileWriter writer = new FileWriter(file)) {
        gson.toJson(new ArrayList<>(merged.values()), writer);
    } catch (IOException e) {
        System.err.println("❌ Failed to write storage file: " + e.getMessage());
    }
}

Engineering for Failure: Per-Record Fault Tolerance

The blessed@com experience taught me a vital lesson. Instead of a global try-catch that would bring down the whole application, I wrapped each individual email send attempt in its own try-catch block inside the processing loop. Any send failure is logged in detail without disrupting the batch.

// From BotRunner.java — per-record fault isolation
private void processNewRecord(ScrapedUser record, String type) {
    if (record.getEmail() == null || record.getEmail().isBlank()) {
        System.out.printf("🤷 Skipping %s — no email address.%n", record.getFirstName());
        return;
    }
    if (!emailValidationService.isEmailValid(record.getEmail())) {
        System.out.printf("🤷 Skipping %s (%s) — email failed validation.%n",
            record.getFirstName(), record.getEmail());
        return;
    }
    try {
        String name = (record.getFirstName() == null || record.getFirstName().isBlank())
            ? "there" : record.getFirstName();
        String templateName = "business".equals(type)
            ? "business_approval_template.html"
            : "user_welcome_template.html";
        String body = String.format(templateService.getTemplate(templateName), name);
        String subject = "business".equals(type)
            ? "Your Business is Now Live on Fixme! 🎉"
            : "Welcome to Fixme! 🎉";
        emailService.sendEmail(record.getEmail(), subject, body);
    } catch (Exception e) {
        System.err.printf("❌ Failed to process %s (%s): %s%n",
            record.getFirstName(), record.getEmail(), e.getMessage());
        // Loop continues unharmed to the next record
    }
}

HTML Templates as Classpath Resources

Rather than embedding HTML as unwieldy string constants in Java code, the email templates live as proper files under src/main/resources/templates/. TemplateService loads them at runtime using Spring's classpath resource API, keeping the Java classes clean and making the templates independently editable.

// From TemplateService.java
public String getTemplate(String templateName) throws IOException {
    try (var reader = new InputStreamReader(
            new ClassPathResource("templates/" + templateName).getInputStream(),
            StandardCharsets.UTF_8)) {
        return FileCopyUtils.copyToString(reader);
    }
}

Admin Reporting and Test Mode

After every job run, BotRunner sends a structured summary email to the admin address configured in application.properties. If the job completes successfully, the report details how many new users and businesses were found and processed. If the job throws an unexpected exception, a failure report is sent with the error details — so no run ever goes unobserved.

For safe testing without touching production users, the bot supports a test-email Spring profile. When active, instead of running the full scraping job, it sends both email templates to the admin inbox using placeholder names, confirming that templates, SMTP connections, and mail config are all working correctly before a live run.

The Brevo Integration: Built for What Comes Next

One of the more forward-thinking additions is PromotionalEmailService. While the bot currently delivers all emails via SMTP for its reliability and simplicity, a production marketing system eventually needs delivery analytics: open rates, click tracking, bounce detection. PromotionalEmailService wraps the official Brevo SDK v7.0.0, providing a fully wired sendTrackableEmail() method that can swap in for SMTP delivery the moment we need those insights — no refactoring required.

The Final Result: A Core Business Process, Engineered from Nothing

The impact was immediate and measurable. What follows isn't a list of features — it's a description of a business process that now simply exists, where before there was silence.

The dual-SMTP failover, per-record fault tolerance, and DNS-level email validation collectively ensure the bot completes its mission even when individual records are malformed or a mail server is temporarily unreachable. The nine-class modular architecture and pre-built Brevo integration mean that adding open-rate tracking, WhatsApp Business API delivery, or new user segments requires adding to the system, not rewriting it.

Ultimately, this project reinforced my core belief that the most impactful work often lies in solving the unglamorous problems that everyone else has accepted as a fact of life. By taking complete ownership — from business strategy and creative design, to deep engineering with Java 17, Spring Boot, and AI-assisted tooling — I was able to single-handedly build a solution where none existed.

I didn't just fix a leaky funnel. I engineered the system that ensures, for every single customer who joins our platform, their journey begins with a truly magic moment.