Backend Dev – Caching, Async Work and Messaging

A request that takes 40ms feels instant. One that takes 4 seconds feels broken. Three tools close that gap, and each has a distinct cost:

• Caching — do not compute it again. Cost: it can be stale.
• Async — do it, but not while the caller waits. Cost: no result to return.
• Messaging — hand it to a different process entirely. Cost: real infrastructure, and a message can be delivered twice.

Before any of them: measure

Most slow endpoints are slow for a boring reason — a missing index, or the N+1 query from the last post. Caching a query that takes 2 seconds because it has no index gives you a fast wrong answer some of the time and a 2-second answer the rest.

Find out where the time goes first. The demo app does it with an aspect that times every service call and logs the slow ones — forty methods instrumented by one class:

@Around("serviceLayer() || reportLayer()")
public Object time(ProceedingJoinPoint joinPoint) throws Throwable {
    long start = System.nanoTime();
    String target = joinPoint.getSignature().toShortString();

    try {
        return joinPoint.proceed();
    } catch (Throwable ex) {
        long millis = (System.nanoTime() - start) / 1_000_000;
        log.warn("{} failed after {} ms: {}", target, millis, ex.toString());
        throw ex;   // never swallow it
    } finally {
        long millis = (System.nanoTime() - start) / 1_000_000;
        if (millis >= SLOW_CALL_MILLIS) {
            log.warn("SLOW {} took {} ms", target, millis);
        }
    }
}

Caching: what is actually safe to cache

Something is a good candidate when it is read far more often than it changes and identical for everybody. A menu qualifies: every visitor loads it, it is the same for all of them, and it changes only when an administrator edits the catalogue.

A cart does not. Neither does an order. They are per-user and change constantly, and caching them would be a correctness bug dressed as an optimisation.

@Override
@Cacheable(value = CacheConfig.MENU_CACHE, keyGenerator = "methodAwareKeyGenerator")
@Transactional(readOnly = true)
public List<ProductDTO> getMenu() {
    return mapper.mapProductsToProductDTOs(productDAO.findActiveMenu());
}

@Override
@Caching(
        evict = {
            @CacheEvict(value = CacheConfig.MENU_CACHE, allEntries = true),
            @CacheEvict(value = CacheConfig.MENU_BY_TYPE_CACHE, allEntries = true)
        })
@Transactional
public ProductDTO createProduct(ProductCreateDTO dto) { ... }

Reads populate the cache; every write clears it. That is the simplest invalidation strategy there is and it is the right default — evict broadly rather than trying to surgically update one entry. Menu edits are rare, so throwing the whole thing away costs nothing.

The bug this design is avoiding

Notice the cache names are public static final constants, not strings.

/** The active, customer-facing menu. */
public static final String MENU_CACHE = "menu";

/** The active menu filtered to one product type. */
public static final String MENU_BY_TYPE_CACHE = "menuByType";

@Cacheable and @CacheEvict refer to a cache by name. Spell it differently in the two places — reads populate "menu", writes evict "menus" — and stale data is served forever with nothing failing anywhere. Constants make the compiler enforce the agreement.

The gotcha nobody warns you about

An in-memory cache is per process. That is fine on one instance and wrong the moment there are two:

/**
 * ⚠️ ConcurrentMapCacheManager is a single-JVM cache
 *
 * It is an in-memory ConcurrentHashMap: nothing evicts by time, nothing bounds its size,
 * and each instance has its own copy. That is fine for one process and wrong the moment
 * there are two — instance A evicts on a write and instance B keeps serving the old menu
 * until it restarts. A real deployment points spring.cache.type at Redis or Caffeine and
 * gets a shared cache or a TTL.
 */

The admin edits the menu, sees the change on instance A, and half the customers keep seeing the old menu because the load balancer sent them to instance B. Two fixes: a shared cache (Redis) so there is only one copy, or a TTL so every copy is wrong for at most a few seconds. Setting a TTL even when you think you do not need one is cheap insurance — it turns "wrong forever" into "wrong briefly". See Spring Boot – Caching and AWS – Elasticache.

Async: get the work off the request thread

Placing an order should not wait for a confirmation email. The email might take two seconds, and the mail server might be down — neither has anything to do with whether the order succeeded.

Moving it off the request thread needs a pool to move it to, and the settings matter:

@Bean(name = "taskExecutor")
public ThreadPoolTaskExecutor taskExecutor() {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(25);
    executor.setMaxPoolSize(150);
    executor.setQueueCapacity(100);
    executor.setThreadNamePrefix("Pizza-API-");
    executor.setWaitForTasksToCompleteOnShutdown(true);
    executor.setAwaitTerminationSeconds(60);
    executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
    executor.initialize();
    return executor;
}

Two of those lines are the ones people leave out and regret. waitForTasksToCompleteOnShutdown means a deploy does not kill work in flight. CallerRunsPolicy decides what happens when the queue is full: instead of throwing the task away, the submitting thread runs it — which slows the caller down and thereby stops new work arriving faster than it can be done. Backpressure, in one line.

The two-annotation combination worth learning

Here is the confirmation email in the demo app. Read the annotations before the body:

@Async
@TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
public void sendConfirmation(OrderPlacedEvent event) {
    try {
        mailService.sendOrderConfirmation(orderService.getOrderByPublicId(event.orderPublicId()));
    } catch (Exception ex) {
        log.error("Confirmation for order {} failed — the order itself is unaffected",
                  event.orderPublicId(), ex);
    }
}

The two annotations do different jobs and you usually need both:

• AFTER_COMMIT decides when — not until the database transaction has actually committed.
• @Async decides on which thread — not the one the customer is waiting on.

Get the first one wrong and the bug is nasty precisely because it is rare: a plain listener sends the email while the transaction is still open, so a rollback a millisecond later leaves the customer holding a confirmation for an order that no longer exists. It works in every test until the one time the transaction fails.

The try/catch is also deliberate. An exception escaping an @Async void method is invisible to the caller — it goes to an uncaught-exception handler and nowhere near the customer, who has already been told the order succeeded. Which it did. A failed confirmation email is not a failed order, and the method has to behave that way.

Events: decoupling inside one process

Without events, the method that creates an order ends with a growing list of calls — the kitchen service, the email service, the analytics service. Each is a new dependency injected into a class whose job is to create an order, and each is a new way for an unrelated failure to break checkout.

Publishing one event inverts that. The order code announces what happened and stops caring who listens:

events.publishEvent(OrderPlacedEvent.from(saved));

One rule about the event itself: carry values, not entities.

/**
 * Carry values, not entities. This record holds the id and the totals rather than the
 * CustomerOrder itself. An async listener runs on a different thread, after the transaction
 * and its persistence context have closed, so touching a lazy association on a detached
 * entity there throws LazyInitializationException.
 */
public record OrderPlacedEvent(
        UUID orderPublicId, String contactEmail, BigDecimal total, OrderType orderType) {}

Messaging: decoupling between processes

An event dies with the process. A message on a broker — ActiveMQ, RabbitMQ, Kafka, SQS — survives it, and can be consumed by an application written by another team in another language.

Reach for a queue when work must outlive the request, when the consumer is a different service, or when you need to absorb a spike by letting a backlog form instead of falling over.

Two pieces of infrastructure that beginners skip and then need:

• A dead-letter queue. Without somewhere for poison messages to go, a message that always fails is redelivered forever: it blocks the queue, burns CPU, fills the log, and hides the real bug behind the noise.
• An explicit wire format. Out of the box JMS moves a String, byte[], Serializable or Map. Push a domain object through that and you either get a failure or you drag Java serialisation into your wire format, which couples both ends of the queue to your class files. Configure a JSON converter instead: the contract stays readable and the consumer can be written in something other than Java.

And publish after the commit, for the same reason as the email — a consumer that reads the order from the database before the insert is visible will act on an order that a rollback is about to erase. Message brokers have no idea your transaction exists.

Retries, and the thing they will do to you

Anything crossing the network fails sometimes, so you retry. Two rules make retries safe rather than dangerous.

Back off. Retrying immediately, from every instance at once, is how a struggling service is turned into a dead one. Wait 1s, then 2s, then 4s, with a little randomness so the retries do not all land together.

Make the operation idempotent, or do not retry it. This is the important one. A timeout does not tell you whether the other side acted — it may have charged the card and lost the response. Retrying blindly charges twice. The fix is an idempotency key: the caller generates a unique id per attempt, you record it, and a second request with the same key returns the first result instead of doing the work again. Every payment provider works this way, and it is worth copying for any operation with a real-world effect.

Finally, decide what happens when a background task fails permanently. The demo app's answer is explicit and worth stealing:

} catch (Exception ex) {
    // The order is already committed and paid for. A broker outage must not turn that into
    // a customer-visible failure, so this is logged and swallowed — the same call the
    // confirmation email makes.
    log.error("Could not publish order {} — the order is unaffected", message.orderId(), ex);
}

Swallowing an exception is normally a mistake. Here it is a decision, and the comment says which one. That is the difference.

What to remember

• Measure before optimising. Most slow endpoints are a missing index or an N+1.
• Cache what is read often, changes rarely and is the same for everyone. Never per-user state.
• Name caches with constants; evict broadly on write.
• An in-memory cache is per process. Two instances means two answers — use Redis or a TTL.
• AFTER_COMMIT decides when; @Async decides on which thread. You usually need both.
• Events carry values, never entities.
• A queue when work must outlive the request or cross a service boundary. Give it a dead-letter queue and a JSON contract.
• Retries need backoff and idempotency. A timeout does not mean it did not happen.

Next: testing — how you change code you did not write without being afraid.