Accelerating Java and Material-UI CI Pipelines Through Podman Multi-Stage Builds and Cache Management

The CI pipeline for our full-stack monorepo was consistently taking over 20 minutes to complete. This wasn’t for a particularly complex application: a standard Java 17 Spring Boot backend serving a REST API and a React frontend built with Material-UI for the component library. The friction was palpable. Every minor frontend tweak or backend logic change triggered this lengthy, resource-intensive process. The initial setup was straightforward, but in a real-world project, developer velocity is a critical metric, and this pipeline was a lead weight tied to it. The core problem was a naive containerization strategy that treated every build as a clean slate, ignoring decades of dependency management and caching wisdom.

Our initial Containerfile was a single-stage behemoth. It encapsulated the entire build process, but at a tremendous cost.

# DO NOT USE THIS - DEMONSTRATION OF THE INEFFICIENT APPROACH
FROM maven:3.8.5-openjdk-17 as builder

# Set up the workspace
WORKDIR /app

# Copy the entire monorepo source
COPY . .

# Build the Java backend
RUN mvn clean package -pl backend -am

# Build the frontend
RUN npm install --prefix frontend
RUN npm run build --prefix frontend

# Copy the frontend build artifacts to the backend's static resources
RUN cp -r frontend/build/ ./backend/src/main/resources/static

# Re-package the backend with the frontend artifacts
RUN mvn package -pl backend -am

# --- Final Stage ---
FROM openjdk:17-slim

WORKDIR /app

# Copy the fat JAR from the builder stage
COPY --from=builder /app/backend/target/backend-0.0.1-SNAPSHOT.jar app.jar

EXPOSE 8080

ENTRYPOINT ["java", "-jar", "app.jar"]

On paper, it works. In practice, it’s a disaster for CI. Every run, regardless of what changed, would re-download every Maven dependency and every npm package. The context sent to the Podman daemon was the entire monorepo. A documentation change in a README.md would trigger a full, uncached rebuild. The resulting image was also bloated with build-time dependencies and intermediate artifacts. This had to be fixed. The move to Podman was a deliberate choice for its daemonless architecture and improved security posture with rootless containers, but its build principles are fundamentally similar to Docker’s. The solution lay not in the tool, but in how we instructed it to build.

The migration to an optimized build process centered on a multi-stage Containerfile and a CI strategy that treated our container registry as a distributed cache layer. This is not a novel concept, but its application requires careful orchestration of dependency management and build stages to maximize cache hits.

First, we deconstructed the build into logical, independent stages. The key principle is to order instructions from least to most frequently changing. For a Java project, the pom.xml changes far less often than the source code. For a Node.js project, package.json is similarly stable.

graph TD
    subgraph Build Pipeline
        A(Start Build) --> B{Code Changed?};
        B --> |pom.xml| C[Invalidate Maven Deps Layer];
        B --> |Java Source| D[Invalidate Source Layer];
        B --> |package.json| E[Invalidate npm Deps Layer];
        B --> |React Source| F[Invalidate Source Layer];

        subgraph "Stage 1: Backend Builder"
            G(Base Maven Image) --> H(Copy pom.xml);
            H --> I(Run mvn dependency:go-offline);
            I --> J(Copy Backend Source);
            J --> K(Run mvn package);
        end

        subgraph "Stage 2: Frontend Builder"
            L(Base Node Image) --> M(Copy package.json);
            M --> N(Run npm install);
            N --> O(Copy Frontend Source);
            O --> P(Run npm run build);
        end

        subgraph "Stage 3: Final Image"
            Q(Base JRE Image) --> R(Copy JAR from Stage 1);
            R --> S(Copy Static Assets from Stage 2);
            S --> T(Final Image Ready);
        end

        C --> I;
        D --> K;
        E --> N;
        F --> P;

        K --> R;
        P --> S;
    end

This flow visualizes how changes selectively invalidate stages, preserving cached layers from previous builds. The implementation of this logic resides entirely within the revised Containerfile.

# Stage 1: Build the Java Backend
# We use a specific version tag to ensure build reproducibility.
FROM maven:3.8.7-openjdk-17 AS backend-builder
WORKDIR /app/backend

# This is the first critical optimization. By copying only the pom.xml first,
# we create a layer that only changes when our dependencies change.
# 'mvn dependency:go-offline' downloads all dependencies without compiling.
# Subsequent builds will use the cache for this layer if pom.xml is unchanged.
COPY backend/pom.xml ./pom.xml
RUN mvn dependency:go-offline

# Now, copy the source code. A change here will only invalidate this
# layer and the subsequent build layer, not the dependency layer.
COPY backend/src/ ./src/

# Build the application. The -DskipTests flag is crucial for CI builds;
# tests should run in a separate, dedicated stage.
RUN mvn package -DskipTests

# ---

# Stage 2: Build the React/MUI Frontend
FROM node:18.18-alpine AS frontend-builder
WORKDIR /app/frontend

# Similar to the Maven step, we first copy package manifests.
# This creates a stable layer for our node_modules directory.
COPY frontend/package.json frontend/package-lock.json ./
RUN npm ci --loglevel warn

# Copy the frontend source code.
COPY frontend/ ./

# Generate the production build.
RUN npm run build

# ---

# Stage 3: Create the Final Production Image
# We use a slim JRE image to minimize the final size. No build tools are needed.
FROM eclipse-temurin:17-jre-focal
WORKDIR /app

# Define arguments for user/group to run the application as non-root.
# A common mistake is running containers as root, which is a security risk.
ARG APP_USER=appuser
ARG APP_GROUP=appgroup
ARG UID=1001
ARG GID=1001

# Create the group and user
RUN groupadd -g ${GID} ${APP_GROUP} && \
    useradd -u ${UID} -g ${APP_GROUP} -s /bin/sh ${APP_USER}

# Copy the built backend artifact from the 'backend-builder' stage.
COPY --from=backend-builder /app/backend/target/*.jar app.jar

# Copy the compiled frontend assets into the Spring Boot static resources directory.
# Spring Boot will automatically serve files from 'classpath:/static/'.
COPY --from=frontend-builder --chown=${APP_USER}:${APP_GROUP} /app/frontend/build /static/

# Change ownership of the application jar
RUN chown ${APP_USER}:${APP_GROUP} app.jar

# Switch to the non-root user
USER ${APP_USER}

# Expose the application port
EXPOSE 8080

# Define the entry point
ENTRYPOINT ["java", "-jar", "-Dserver.port=8080", "app.jar"]

This Containerfile is vastly superior. It produces a smaller, more secure image and enables effective layer caching. However, it only solves half the problem. In a typical CI environment where runners are ephemeral, the local Podman storage and build cache are destroyed after each job. The next build starts from scratch, re-pulling base images and re-building every layer.

The solution is to use our container registry not just as an artifact repository, but as a persistent, distributed build cache. The podman build command supports the --cache-from flag, which instructs the builder to use a specified image as a source for cache layers. We configure our CI pipeline to pull the most recently built image (latest tag) and use it as a cache source. After a successful build, the new image is pushed with both its unique commit tag and the latest tag, effectively propagating the cache for the next run.

Here is a practical implementation using a GitLab CI pipeline:

# .gitlab-ci.yml
# This pipeline demonstrates the optimized build and push process.

variables:
  # Define the image name and registry location.
  # CI_REGISTRY_IMAGE is a predefined GitLab variable.
  IMAGE_NAME: $CI_REGISTRY_IMAGE
  IMAGE_TAG: $CI_COMMIT_SHORT_SHA
  LATEST_TAG: latest

stages:
  - build
  - test # Placeholder for unit/integration test stages
  - release

build-and-push:
  stage: build
  # We use the official Podman image to run our build commands.
  # The 'dind' (Docker-in-Docker) service is not needed with Podman's architecture.
  image: quay.io/podman/stable
  
  # Login to the GitLab container registry.
  # Predefined CI variables are used for credentials.
  before_script:
    - podman login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
  
  script:
    # 1. Attempt to pull the 'latest' image to use as a cache source.
    # The '|| true' ensures the job doesn't fail if the image doesn't exist yet (e.g., first run).
    - podman pull $IMAGE_NAME:$LATEST_TAG || true

    # 2. Build the image.
    # --cache-from: Use the pulled 'latest' image as a cache source.
    # --tag: Apply the commit-specific tag.
    # --tag: Also apply the 'latest' tag for the next build's cache.
    # The build command is logged for debugging purposes.
    - >
      podman build
      --format=docker
      --cache-from $IMAGE_NAME:$LATEST_TAG
      --tag $IMAGE_NAME:$IMAGE_TAG
      --tag $IMAGE_NAME:$LATEST_TAG
      .

    # 3. Push both tags to the registry.
    - podman push $IMAGE_NAME:$IMAGE_TAG
    - podman push $IMAGE_NAME:$LATEST_TAG

  rules:
    # This job only runs for changes on the main branch.
    - if: '$CI_COMMIT_BRANCH == "main"'

This CI configuration is the lynchpin. It orchestrates the pull-build-push cycle that turns the registry into a cache. When only a single Java file changes, the pipeline executes as follows:

1. Pulls the latest image.
2. Starts the podman build process.
3. The backend-builder stage hits the cache for the pom.xml and mvn dependency:go-offline layers.
4. The COPY backend/src/ layer is invalidated. It’s re-executed.
5. The mvn package layer is re-executed.
6. The frontend-builder stage finds that package.json and the frontend source are unchanged, so it uses the cached layers for npm ci and npm run build completely.
7. The final stage reassembles the new JAR and the cached frontend assets.
8. The new image is pushed.

The result is a build that takes 3-5 minutes instead of 20+. The majority of time is spent only on the component that actually changed: recompiling the Java code.

To make this concrete, let’s look at the application code this pipeline is building. It’s not “hello world”; it’s structured to resemble a production service.

Backend: src/main/java/com/example/app/api/DataController.java

package com.example.app.api;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import java.time.Instant;
import java.util.Map;

@RestController
@RequestMapping("/api/data")
public class DataController {

    private static final Logger logger = LoggerFactory.getLogger(DataController.class);

    @GetMapping("/status")
    public ResponseEntity<Map<String, Object>> getStatus() {
        // In a real-world project, this would involve more complex logic,
        // such as checking database connections or downstream service health.
        logger.info("Status endpoint invoked at {}", Instant.now());

        try {
            Map<String, Object> response = Map.of(
                "status", "OK",
                "timestamp", Instant.now().toEpochMilli(),
                "serviceVersion", "1.0.2" // Example static data
            );
            return ResponseEntity.ok(response);
        } catch (Exception e) {
            // Basic error handling for unexpected issues.
            logger.error("Failed to retrieve application status", e);
            Map<String, Object> errorResponse = Map.of(
                "status", "ERROR",
                "message", "An internal error occurred."
            );
            return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).body(errorResponse);
        }
    }
}

Frontend: src/components/StatusDisplay.jsx

import React, { useState, useEffect } from 'react';
import { Card, CardContent, Typography, Button, CircularProgress, Alert, Box } from '@mui/material';

const StatusDisplay = () => {
  const [statusData, setStatusData] = useState(null);
  const [loading, setLoading] = useState(true);
  const [error, setError] = useState(null);

  const fetchData = async () => {
    setLoading(true);
    setError(null);
    try {
      // The '/api' prefix will be proxied to the backend by the dev server,
      // and served directly by Spring Boot in production.
      const response = await fetch('/api/data/status');
      if (!response.ok) {
        throw new Error(`HTTP error! status: ${response.status}`);
      }
      const data = await response.json();
      setStatusData(data);
    } catch (e) {
      console.error("Failed to fetch status:", e);
      setError(e.message);
    } finally {
      setLoading(false);
    }
  };

  useEffect(() => {
    fetchData();
  }, []);

  return (
    <Box sx={{ p: 2, maxWidth: 500, margin: 'auto' }}>
      <Card variant="outlined">
        <CardContent>
          <Typography variant="h5" component="div" gutterBottom>
            Application Status
          </Typography>
          {loading && <CircularProgress />}
          {error && <Alert severity="error">Failed to load status: {error}</Alert>}
          {statusData && !loading && (
            <Box>
              <Typography sx={{ mb: 1.5 }} color="text.secondary">
                Status: {statusData.status}
              </Typography>
              <Typography variant="body2">
                Version: {statusData.serviceVersion}
              </Typography>
              <Typography variant="caption" display="block" color="text.secondary">
                Timestamp: {new Date(statusData.timestamp).toLocaleString()}
              </Typography>
            </Box>
          )}
          <Button variant="contained" onClick={fetchData} disabled={loading} sx={{ mt: 2 }}>
            Refresh
          </Button>
        </CardContent>
      </Card>
    </Box>
  );
};

export default StatusDisplay;

This setup provides a robust foundation. The performance gains are not just theoretical; they translate directly into faster feedback cycles for developers and lower CI resource consumption. A common pitfall here is cache poisoning. If a build introduces a corrupted dependency into the latest cache image, subsequent builds might fail mysteriously. The remediation is to manually delete the latest tag from the registry, forcing a full, clean rebuild, and then investigate the root cause. This is a rare but important operational consideration.

The presented solution is a significant step up from a naive build process, but it is not the final word on build optimization. For massive monorepos with hundreds of microservices or libraries, this layer-based caching can become insufficient. A change to a core shared library would still trigger a rebuild of all dependent services. In those scenarios, more advanced build systems like Bazel, which build a fine-grained dependency graph and offer remote caching of individual build artifacts, become necessary. However, for a vast number of projects, the multi-stage Podman build coupled with a registry-as-cache strategy strikes an effective balance between implementation complexity and performance improvement. It leverages native container tooling without introducing a complex new build system, making it a pragmatic choice for accelerating development.