Subscribe To Our Newsletter
You will receive our latest post and tutorial.
Thank you for subscribing!

required
required


MySQL Closure Table

Sometimes we are given a task where we need to model a hierarchy of different complexity levels and not really sure how to do that properly in the most efficient, reliable, and flexible way. Let’s review one of the data modeling patterns that give us some answers for that.

Consider we have a hierarchy with a ragged variable depth, like on a picture below.

The requirements are that we need to:

  • Query a subtree, preferably up to a certain level of depth, like direct subordinate as well as descendants up to a certain level
  • Query all ancestors of the node
  • Add/Remove a node to a hierarchy
  • Change the layout of the hierarchy — move a subtree from one place to another (from one parent to another one)
  • Manage efficiently the forest —a large number of trees, where each tree is not necessarily big in terms of number of nodes, and the maximum depth, but the number of trees could grow up to hundreds of thousands or even millions
  • Manage nodes of the trees that enter short-term relationships. So there is a need to track the time period for which the actual relationships took place.

This is where a closure table comes into the picture.

A closure table gives you the ability to work with tree structure hierarchies. It involves storing all paths through a tree, not just those with a direct parent-child relationship. For example: things like family tree(Parent – Kids – etc), company hierarchy(Owner – CEO – Manager – Employee), or file sytem(/users/folauk/Documents/test.mp4). There are other ways such as Path Enumeration and Nested Sets to keep track of this kind of information but using a closure is the simplest and most efficient way of doing it.

The main building block of the pattern is an additional structure(table) or a mapping table that keeps the tree relationships — a set of node pairs for each path from the ancestor to a descendant or parent to child, including a path to itself.

 

Our example here is a social media site where a user can put up a post. Other users can comment on that post and can comment on comments from the same post. The tree here is comments under comments. We have to keep track of which comments go under which comments and their parents.

CREATE TABLE `user` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `date_of_birth` date DEFAULT NULL,
  `email` varchar(255) DEFAULT NULL,
  `first_name` varchar(255) DEFAULT NULL,
  `gender` varchar(255) DEFAULT NULL,
  `last_name` varchar(255) DEFAULT NULL,
  `phone_number` varchar(255) DEFAULT NULL,
  PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=latin1;

CREATE TABLE `post` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `content` longtext,
  `created_at` datetime(6) DEFAULT NULL,
  `last_updated_at` datetime(6) DEFAULT NULL,
  `uuid` varchar(255) NOT NULL,
  `author_id` bigint(20) DEFAULT NULL,
  PRIMARY KEY (`id`),
  UNIQUE KEY `UK_b1a5vs6t5vi32stck5kjw0tgf` (`uuid`),
  KEY `FK12njtf8e0jmyb45lqfpt6ad89` (`author_id`),
  CONSTRAINT `FK12njtf8e0jmyb45lqfpt6ad89` FOREIGN KEY (`author_id`) REFERENCES `user` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=41 DEFAULT CHARSET=latin1;

CREATE TABLE `comment` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `content` longtext,
  `created_at` datetime(6) DEFAULT NULL,
  `last_updated_at` datetime(6) DEFAULT NULL,
  `author_id` bigint(20) DEFAULT NULL,
  `post_id` bigint(20) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `FKh1gtv412u19wcbx22177xbkjp` (`author_id`),
  KEY `FKs1slvnkuemjsq2kj4h3vhx7i1` (`post_id`),
  CONSTRAINT `FKh1gtv412u19wcbx22177xbkjp` FOREIGN KEY (`author_id`) REFERENCES `user` (`id`),
  CONSTRAINT `FKs1slvnkuemjsq2kj4h3vhx7i1` FOREIGN KEY (`post_id`) REFERENCES `post` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=201 DEFAULT CHARSET=latin1;

CREATE TABLE `comment_tree` (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `depth` int(11) DEFAULT NULL,
  `child_id` bigint(20) DEFAULT NULL,
  `parent_id` bigint(20) DEFAULT NULL,
  PRIMARY KEY (`id`),
  KEY `FKhosfyxi6cpykkhyrccfxq2k2x` (`child_id`),
  KEY `FKq2283664y5ywd961iojgjk98b` (`parent_id`),
  CONSTRAINT `FKhosfyxi6cpykkhyrccfxq2k2x` FOREIGN KEY (`child_id`) REFERENCES `comment` (`id`),
  CONSTRAINT `FKq2283664y5ywd961iojgjk98b` FOREIGN KEY (`parent_id`) REFERENCES `comment` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1201 DEFAULT CHARSET=latin1;

Here we have a comment table that represents our comment, author of the comment, and the post it belongs to.

Insert comments(rows) into the comment_tree(closure) table

INSERT INTO comment_tree(parent_id, child_id, depth) 
SELECT p.parent_id, c.child_id, p.depth+c.depth+1 
FROM comment_tree as p, comment_tree as c 
WHERE p.child_id = :parentId and c.parent_id = :childId;

Delete comments(rows) in the comment_tree(closure) table

DELETE link
FROM comment_tree as parent, comment_tree as link, comment_tree as child
WHERE parent.parent_id = link.parent_id AND child.child_id = link.child_id
-- AND parent.child_id = {parentId} and child.parent_id = {childId}
AND parent.child_id = ? AND child.parent_id = ?

Retrieve comments under a comment.

Get all comments under comment with id 4.

SELECT com.*
FROM comment AS com
JOIN comment_tree AS comTree ON com.id = comTree.child_id
WHERE comTree.parent_id = 4;

Retrieve parent comments of a comment

Get all parent comments of comment with id 4.

-- retrieve parent comments of comment with id 4
SELECT com.*, comTree.*
FROM comment AS com
JOIN comment_tree AS comTree ON com.id = comTree.parent_id
WHERE comTree.child_id = 4;

Source code on Github

September 29, 2018

Java Best Practices




Writing Java code that compiles and runs is easy. Writing Java code that is readable, maintainable, secure, and performant — code that your teammates (and your future self) will thank you for — is a different challenge entirely. This tutorial distills the practices that separate production-quality Java from weekend-project Java.

We will cover 20 best practices organized into seven categories: code style, object-oriented design, error handling, performance, modern Java features, testing and quality, code organization, and security. Each practice includes concrete bad and good examples so you can immediately apply what you learn.



Code Style and Readability

You read code far more often than you write it. Studies consistently show that developers spend 70-80% of their time reading existing code. The practices in this section make that reading time dramatically more productive.

1. Naming Conventions

Names are the most important form of documentation in your code. A well-chosen name eliminates the need for comments, makes code self-documenting, and reduces the cognitive load on every developer who reads it. Java has established conventions that every professional project follows:

Element Convention Example
Classes and Interfaces PascalCase (nouns) UserAccount, PaymentProcessor
Methods camelCase (verbs) calculateTotal(), sendNotification()
Variables camelCase (nouns) firstName, orderCount
Constants UPPER_SNAKE_CASE MAX_RETRY_COUNT, DEFAULT_TIMEOUT
Packages all lowercase, dot-separated com.example.payment
Type Parameters Single uppercase letter T, E, K, V
Enums PascalCase (type), UPPER_SNAKE_CASE (values) OrderStatus.PENDING

Boolean names deserve special attention. A boolean variable or method should read like a true/false question:

// BAD - unclear what these represent
boolean flag = true;
boolean status = false;
boolean check = user.verify();

// GOOD - reads like a question: "Is it active?" -> true/false
boolean isActive = true;
boolean hasPermission = false;
boolean canExecute = user.hasRole("ADMIN");
boolean shouldRetry = attempts < MAX_RETRIES;
boolean wasProcessed = order.getStatus() == Status.COMPLETE;

Avoid abbreviations. Abbreviations save a few keystrokes but cost minutes of confusion every time someone reads the code. Modern IDEs have autocomplete -- there is no reason to abbreviate.

// BAD - abbreviations force the reader to decode
int qty;
String addr;
UserAcctMgr uam;
public void calcTtlAmt() { ... }

// GOOD - instantly clear
int quantity;
String address;
UserAccountManager userAccountManager;
public void calculateTotalAmount() { ... }

Use meaningful, intention-revealing names. The name should tell you why it exists, what it does, and how it is used. If a name requires a comment, the name is wrong.

// BAD - what does d represent? What are the 86400000?
long d = System.currentTimeMillis() - start;
if (d > 86400000) { ... }

// GOOD - the name tells the whole story
long elapsedTimeMillis = System.currentTimeMillis() - startTimeMillis;
long ONE_DAY_IN_MILLIS = 24 * 60 * 60 * 1000L;
if (elapsedTimeMillis > ONE_DAY_IN_MILLIS) { ... }

Avoid single-letter variables except in short loop counters (i, j, k) or lambda parameters where the type makes the meaning obvious (e.g., users.forEach(u -> u.activate())).

2. Code Formatting

Consistent formatting is not about personal preference -- it is about team velocity. When everyone follows the same formatting rules, code reviews focus on logic instead of style debates, and every file is instantly familiar.

Key formatting rules:

  • Indentation: Use 4 spaces (not tabs). This is the Java standard.
  • Line length: Keep lines under 120 characters. Long lines force horizontal scrolling and break readability.
  • Braces: Always use braces for if, for, while, even for single statements. This prevents bugs when someone adds a second line.
  • Blank lines: Use blank lines to separate logical blocks within a method, between methods, and between field groups.
  • One statement per line: Never chain multiple statements on a single line.
// BAD - missing braces, inconsistent spacing, dense
if(active) doSomething();
for(int i=0;i<10;i++) process(i);
if (x > 0)
    doThis();
    doThat(); // BUG: doThat() runs ALWAYS, not inside the if

// GOOD - consistent braces, spacing, and structure
if (active) {
    doSomething();
}

for (int i = 0; i < 10; i++) {
    process(i);
}

if (x > 0) {
    doThis();
    doThat(); // clearly inside the if block
}

Use your IDE's auto-format. In IntelliJ IDEA, press Ctrl+Alt+L (Windows/Linux) or Cmd+Option+L (Mac). In Eclipse, press Ctrl+Shift+F. Configure your team's formatting rules once and share the settings file across the project. Consider adding a .editorconfig file to your repository.

3. Comments and Documentation

The best code needs no comments because the names and structure explain everything. When comments are necessary, they should explain why, not what. If you find yourself writing a comment that explains what the code does, consider renaming variables or extracting a method instead.

// BAD - comments that repeat the code (add nothing)
// Increment counter by 1
counter++;

// Check if user is null
if (user == null) {
    return;
}

// Get the list of users from the database
List users = userRepository.findAll();

// GOOD - comments that explain WHY
// Circuit breaker: stop retrying after 3 failures to avoid
// cascading timeouts to downstream services
if (failureCount >= MAX_FAILURES) {
    return fallbackResponse();
}

// Using insertion order map because the API contract guarantees
// fields are returned in the order they were added
Map fields = new LinkedHashMap<>();

Javadoc for public APIs. Every public class and public method should have Javadoc. This is non-negotiable for library code and strongly recommended for application code.

/**
 * Transfers funds between two accounts within a single transaction.
 *
 * 
Both accounts must be active and belong to the same currency.
 * The transfer is atomic -- if any step fails, both accounts
 * remain unchanged.

 *
 * @param fromAccountId the source account ID
 * @param toAccountId   the destination account ID
 * @param amount        the amount to transfer (must be positive)
 * @return the transaction confirmation with a unique reference ID
 * @throws InsufficientFundsException if the source account balance is too low
 * @throws AccountNotFoundException   if either account does not exist
 * @throws IllegalArgumentException   if amount is zero or negative
 */
public TransactionConfirmation transfer(
        String fromAccountId,
        String toAccountId,
        BigDecimal amount) {
    // implementation
}

TODO and FIXME conventions:

  • // TODO: -- something that needs to be done but is not urgent
  • // FIXME: -- a known bug or problematic code that needs fixing
  • // HACK: -- a workaround that should be replaced with a proper solution

Always include your name or ticket number so someone can follow up: // TODO(JIRA-1234): Replace with batch API when available



Object-Oriented Design

Java is fundamentally an object-oriented language. These design practices help you write code that is flexible, testable, and resilient to change.

4. Follow SOLID Principles

SOLID is a set of five design principles that guide you toward maintainable object-oriented code. Here is a quick reminder of each with a practical Java example:

Principle Meaning One-Line Summary
Single Responsibility A class should have only one reason to change Separate UserValidator from UserRepository
Open/Closed Open for extension, closed for modification Add new behavior via new classes, not by editing existing ones
Liskov Substitution Subtypes must be substitutable for their base types If Square extends Rectangle, setting width must not break area calculation
Interface Segregation No client should depend on methods it does not use Split Worker into Workable and Feedable
Dependency Inversion Depend on abstractions, not concretions Constructor takes NotificationService interface, not EmailService class 
// BAD - Single Responsibility violation: one class does everything
public class UserService {
    public void registerUser(User user) { ... }
    public void sendWelcomeEmail(User user) { ... }   // email logic
    public void generatePdfReport(User user) { ... }  // PDF logic
    public boolean validateUser(User user) { ... }     // validation logic
}

// GOOD - each class has one responsibility
public class UserService {
    private final UserValidator validator;
    private final UserRepository repository;
    private final NotificationService notificationService;

    public UserService(UserValidator validator,
                       UserRepository repository,
                       NotificationService notificationService) {
        this.validator = validator;
        this.repository = repository;
        this.notificationService = notificationService;
    }

    public void registerUser(User user) {
        validator.validate(user);
        repository.save(user);
        notificationService.sendWelcomeNotification(user);
    }
}

5. Prefer Composition Over Inheritance

Inheritance creates a tight coupling between parent and child classes. A change in the parent can break every child. Composition -- where a class contains another class rather than extending it -- gives you the same code reuse with much more flexibility.

The rule of thumb: use inheritance only when there is a genuine is-a relationship (Dog is an Animal). If the relationship is has-a or uses-a, use composition.

// BAD - inheritance for code reuse (fragile base class problem)
public class Logger {
    public void log(String message) {
        System.out.println("[LOG] " + message);
    }
}

public class UserService extends Logger {
    // UserService IS-A Logger? That makes no sense.
    public void createUser(String name) {
        log("Creating user: " + name);  // inherited method
        // ...
    }
}

// GOOD - composition: UserService HAS-A Logger
public class UserService {
    private final Logger logger;

    public UserService(Logger logger) {
        this.logger = logger;
    }

    public void createUser(String name) {
        logger.log("Creating user: " + name);
        // ...
    }
}
// Now you can swap FileLogger, ConsoleLogger, or NoOpLogger
// without touching UserService at all.

6. Program to Interfaces

When you declare variables, parameters, and return types, use the most general type that still provides the methods you need. This decouples your code from specific implementations and makes it easy to swap one implementation for another.

// BAD - tied to a specific implementation
ArrayList names = new ArrayList<>();
HashMap userMap = new HashMap<>();

public ArrayList getNames() {
    return names;
}

// GOOD - programmed to the interface
List names = new ArrayList<>();
Map userMap = new HashMap<>();

public List getNames() {
    return names;
}
// Tomorrow you can switch to LinkedList or CopyOnWriteArrayList
// without changing any calling code.

This principle extends to dependency injection. Your classes should depend on interfaces, not concrete classes:

// BAD - hard dependency on a concrete class
public class OrderService {
    private MySQLOrderRepository repository = new MySQLOrderRepository();
}

// GOOD - depends on an abstraction, injected via constructor
public class OrderService {
    private final OrderRepository repository;

    public OrderService(OrderRepository repository) {
        this.repository = repository;
    }
}
// In production: new OrderService(new MySQLOrderRepository())
// In tests:      new OrderService(new FakeOrderRepository())

7. Immutability

An immutable object cannot be modified after creation. Immutable objects are inherently thread-safe, simpler to reason about, and safer to share across your codebase. Make your classes immutable unless there is a compelling reason not to.

To make a class immutable:

  1. Declare the class final (prevent subclassing)
  2. Make all fields private final
  3. Do not provide setter methods
  4. Perform defensive copies of mutable fields in the constructor and getters
// BAD - mutable class with exposed internals
public class Order {
    public List items;  // anyone can modify this list
    public double total;        // anyone can change the total

    public Order(List items) {
        this.items = items;  // stores a reference -- caller can mutate it
    }
}

// GOOD - immutable class with defensive copying
public final class Order {
    private final List items;
    private final double total;

    public Order(List items, double total) {
        // Defensive copy: changes to the original list won't affect us
        this.items = List.copyOf(items);
        this.total = total;
    }

    public List getItems() {
        // Returns an unmodifiable view -- caller cannot add/remove items
        return items; // already unmodifiable from List.copyOf()
    }

    public double getTotal() {
        return total;
    }
}

Use unmodifiable collections when returning collections from methods. This prevents callers from accidentally modifying your internal state.

// Java 9+ factory methods create unmodifiable collections
List colors = List.of("Red", "Green", "Blue");
Set primes = Set.of(2, 3, 5, 7, 11);
Map scores = Map.of("Alice", 95, "Bob", 87);

// For mutable lists you need to return unmodifiable views
public List getUsers() {
    return Collections.unmodifiableList(users);
}

// Java 16+ records are inherently immutable (no setter generation)
public record Money(BigDecimal amount, Currency currency) { }



Error Handling

How you handle errors defines the reliability of your application. Poor error handling leads to silent failures, lost data, and hours of debugging. These practices ensure your application fails predictably and provides useful information when things go wrong.

8. Exception Handling Best Practices

Catch specific exceptions. Catching Exception or Throwable hides bugs, catches things you never intended to handle (like OutOfMemoryError), and makes debugging a nightmare.

// BAD - catches everything, including programming errors
try {
    processOrder(order);
} catch (Exception e) {
    log.error("Something went wrong");  // what went wrong? No clue.
}

// GOOD - catch specific exceptions, handle each appropriately
try {
    processOrder(order);
} catch (InsufficientFundsException e) {
    log.warn("Insufficient funds for order {}: {}", order.getId(), e.getMessage());
    notifyCustomer(order, "Payment failed: insufficient funds");
} catch (InventoryException e) {
    log.error("Inventory check failed for order {}", order.getId(), e);
    queueForRetry(order);
}

Never leave a catch block empty. An empty catch block silently swallows the error and guarantees that someone will spend hours debugging a problem that the application already detected and threw away.

// BAD - the worst thing you can do with an exception
try {
    Files.delete(tempFile);
} catch (IOException e) {
    // silently ignored -- if the delete fails, you'll never know
}

// GOOD - at minimum, log it. Better: handle it.
try {
    Files.delete(tempFile);
} catch (IOException e) {
    log.warn("Failed to delete temp file {}: {}", tempFile, e.getMessage());
}

Use try-with-resources for any object that implements AutoCloseable. This guarantees cleanup even if an exception occurs, and it is more concise than try-finally.

// BAD - manual resource management (verbose and error-prone)
Connection conn = null;
PreparedStatement stmt = null;
try {
    conn = dataSource.getConnection();
    stmt = conn.prepareStatement("SELECT * FROM users WHERE id = ?");
    stmt.setLong(1, userId);
    ResultSet rs = stmt.executeQuery();
    // process results
} catch (SQLException e) {
    log.error("Query failed", e);
} finally {
    if (stmt != null) { try { stmt.close(); } catch (SQLException e) { } }
    if (conn != null) { try { conn.close(); } catch (SQLException e) { } }
}

// GOOD - try-with-resources handles all cleanup automatically
try (Connection conn = dataSource.getConnection();
     PreparedStatement stmt = conn.prepareStatement(
         "SELECT * FROM users WHERE id = ?")) {

    stmt.setLong(1, userId);
    try (ResultSet rs = stmt.executeQuery()) {
        // process results
    }
} catch (SQLException e) {
    log.error("Query failed for user {}", userId, e);
}

Create custom exceptions for business logic. Custom exceptions make your error handling expressive and allow callers to react differently to different business failures.

public class InsufficientFundsException extends RuntimeException {
    private final BigDecimal currentBalance;
    private final BigDecimal requestedAmount;

    public InsufficientFundsException(BigDecimal currentBalance,
                                       BigDecimal requestedAmount) {
        super(String.format("Cannot withdraw %s: current balance is %s",
              requestedAmount, currentBalance));
        this.currentBalance = currentBalance;
        this.requestedAmount = requestedAmount;
    }

    public BigDecimal getCurrentBalance() { return currentBalance; }
    public BigDecimal getRequestedAmount() { return requestedAmount; }
}

// Usage: fail fast with a meaningful exception
public void withdraw(BigDecimal amount) {
    if (amount.compareTo(balance) > 0) {
        throw new InsufficientFundsException(balance, amount);
    }
    balance = balance.subtract(amount);
}

Fail fast. Validate inputs at the beginning of a method. Do not let invalid data travel deep into your code where it causes confusing errors far from the source of the problem.

9. Null Safety

Tony Hoare, who invented null references, called them his "billion-dollar mistake." In Java, NullPointerException is the most common runtime exception. These practices help you avoid it.

Return empty collections instead of null.

// BAD - forces every caller to check for null
public List getOrdersByCustomer(String customerId) {
    List orders = repository.findByCustomerId(customerId);
    if (orders.isEmpty()) {
        return null;  // caller MUST check: if (orders != null)
    }
    return orders;
}

// GOOD - return empty collection, callers can iterate safely
public List getOrdersByCustomer(String customerId) {
    List orders = repository.findByCustomerId(customerId);
    return orders != null ? orders : Collections.emptyList();
}

// Even better with modern Java:
public List getOrdersByCustomer(String customerId) {
    return Objects.requireNonNullElse(
        repository.findByCustomerId(customerId),
        List.of()
    );
}

Use Optional for values that might not exist. Optional is for return types that legitimately may have no value. It forces the caller to handle the absence explicitly.

// BAD - returning null for "not found"
public User findByEmail(String email) {
    // Returns null if not found -- caller might forget to check
    return userMap.get(email);
}

// GOOD - Optional makes the absence explicit
public Optional findByEmail(String email) {
    return Optional.ofNullable(userMap.get(email));
}

// Callers are forced to handle both cases:
userService.findByEmail("alice@example.com")
    .ifPresentOrElse(
        user -> System.out.println("Found: " + user.getName()),
        () -> System.out.println("User not found")
    );

// Or with a default:
User user = userService.findByEmail("alice@example.com")
    .orElseThrow(() -> new UserNotFoundException("alice@example.com"));

Use Objects.requireNonNull() at the start of constructors and methods to catch null early with a clear message.

public class EmailService {
    private final SmtpClient client;
    private final TemplateEngine templateEngine;

    public EmailService(SmtpClient client, TemplateEngine templateEngine) {
        // Fail immediately with a clear message instead of getting a
        // NullPointerException 5 method calls later
        this.client = Objects.requireNonNull(client, "SmtpClient must not be null");
        this.templateEngine = Objects.requireNonNull(templateEngine,
            "TemplateEngine must not be null");
    }
}



Performance and Efficiency

Premature optimization is the root of all evil (Knuth), but knowing the common performance pitfalls saves you from writing obviously slow code. These practices are about choosing the right tool for the job.

10. String Handling

Strings are immutable in Java. Every concatenation creates a new String object. For single-line concatenation the compiler optimizes it, but inside loops you must use StringBuilder.

// BAD - creates a new String object on every iteration
// For 10,000 iterations, this creates ~10,000 intermediate String objects
public String buildReport(List lines) {
    String result = "";
    for (String line : lines) {
        result += line + "\n";  // new String every time
    }
    return result;
}

// GOOD - StringBuilder reuses a single internal buffer
public String buildReport(List lines) {
    StringBuilder sb = new StringBuilder(lines.size() * 80); // pre-size estimate
    for (String line : lines) {
        sb.append(line).append("\n");
    }
    return sb.toString();
}

// ALSO GOOD - String.join() for simple cases
public String buildReport(List lines) {
    return String.join("\n", lines);
}

Text blocks (Java 13+) make multi-line strings readable and maintainable:

// BAD - unreadable concatenation of a multi-line string
String json = "{\n" +
    "  \"name\": \"" + name + "\",\n" +
    "  \"email\": \"" + email + "\"\n" +
    "}";

// GOOD - text block (Java 13+) with formatted() (Java 15+)
String json = """
    {
      "name": "%s",
      "email": "%s"
    }
    """.formatted(name, email);

11. Collections Best Practices

Choosing the right collection is one of the most impactful performance decisions you can make. Here is a quick guide:

Need Use Why
Ordered list with random access ArrayList O(1) get by index, backed by array
Frequent insertions/removals at both ends ArrayDeque O(1) add/remove at head and tail
Unique elements, fast lookup HashSet O(1) contains, add, remove
Sorted unique elements TreeSet O(log n) operations, natural ordering
Key-value lookup HashMap O(1) get/put on average
Sorted key-value pairs TreeMap O(log n) operations, sorted keys
Insertion-ordered key-value pairs LinkedHashMap O(1) operations + maintains order
Thread-safe key-value ConcurrentHashMap Lock striping for concurrent access 
// BAD - wrong collection for the job
LinkedList users = new LinkedList<>();  // random access is O(n)
users.get(500);  // traverses 500 nodes to get element

// GOOD - ArrayList for indexed access
ArrayList users = new ArrayList<>();
users.get(500);  // O(1) array index lookup

// Pre-size when you know the approximate size (avoids resizing)
List names = new ArrayList<>(1000);
Map cache = new HashMap<>(256);

// Use Collections utility methods for common operations
List numbers = Arrays.asList(5, 3, 8, 1, 9);
Collections.sort(numbers);                    // sort in place
int max = Collections.max(numbers);           // find max
Collections.shuffle(numbers);                 // random order
List synced = Collections.synchronizedList(numbers);

Use streams appropriately. Streams are excellent for transformations and filtering but add overhead. Do not use them for simple iterations where a for-each loop is clearer.

// OVERKILL - stream adds nothing for a simple loop
users.stream().forEach(user -> user.sendEmail());

// BETTER - plain for-each is simpler
for (User user : users) {
    user.sendEmail();
}

// GOOD USE OF STREAM - transformation, filtering, collecting
List activeEmails = users.stream()
    .filter(User::isActive)
    .map(User::getEmail)
    .sorted()
    .collect(Collectors.toList());

12. Resource Management

Every resource you open -- database connections, file handles, network sockets, thread pools -- must be closed. Failing to close resources causes memory leaks, connection pool exhaustion, and file handle starvation.

Always use try-with-resources (see Exception Handling section above). Here is a pattern for managing multiple resources:

// Read a file, process each line, write results to another file
public void processFile(Path input, Path output) throws IOException {
    try (BufferedReader reader = Files.newBufferedReader(input);
         BufferedWriter writer = Files.newBufferedWriter(output)) {

        String line;
        while ((line = reader.readLine()) != null) {
            String processed = transform(line);
            writer.write(processed);
            writer.newLine();
        }
    }
    // Both reader and writer are automatically closed here,
    // even if an exception occurred
}

// For custom resources, implement AutoCloseable
public class ConnectionPool implements AutoCloseable {
    private final List connections = new ArrayList<>();

    public Connection acquire() { /* ... */ }

    @Override
    public void close() {
        connections.forEach(conn -> {
            try { conn.close(); } catch (SQLException e) {
                log.warn("Failed to close connection", e);
            }
        });
        connections.clear();
    }
}



Modern Java

Java has evolved rapidly since Java 8. Using modern language features makes your code more concise, readable, and less error-prone. If your project supports Java 17 or later, take advantage of these features.

13. Use Modern Java Features

Local variable type inference -- var (Java 10+): Use var when the type is obvious from the right side. Do not use it when it obscures the type.

// GOOD uses of var - type is obvious from context
var names = new ArrayList();         // clearly ArrayList
var reader = new BufferedReader(new FileReader("data.txt"));
var count = 0;                               // clearly int
var entry = map.entrySet().iterator().next(); // avoids verbose generic type

// BAD uses of var - type is unclear
var result = process();    // what type is result?
var data = getData();      // what type is data?
var x = calculate(a, b);  // impossible to understand without IDE

Records (Java 16+): Use records for simple data carriers. A record automatically generates the constructor, getters, equals(), hashCode(), and toString().

// BAD - 50 lines of boilerplate for a simple data class
public class Point {
    private final int x;
    private final int y;

    public Point(int x, int y) {
        this.x = x;
        this.y = y;
    }

    public int getX() { return x; }
    public int getY() { return y; }

    @Override
    public boolean equals(Object o) { /* 10 lines */ }
    @Override
    public int hashCode() { return Objects.hash(x, y); }
    @Override
    public String toString() { return "Point[x=" + x + ", y=" + y + "]"; }
}

// GOOD - one line with Java 16+ records
public record Point(int x, int y) { }

// Records can have validation and custom methods
public record EmailAddress(String value) {
    public EmailAddress {
        // Compact constructor for validation
        if (value == null || !value.contains("@")) {
            throw new IllegalArgumentException("Invalid email: " + value);
        }
    }
}

Sealed classes (Java 17+): Restrict which classes can extend a type. This enables exhaustive pattern matching in switch expressions.

// Sealed class - only these three can extend Shape
public sealed interface Shape
    permits Circle, Rectangle, Triangle {
}

public record Circle(double radius) implements Shape { }
public record Rectangle(double width, double height) implements Shape { }
public record Triangle(double base, double height) implements Shape { }

// The compiler KNOWS all subtypes, so switch can be exhaustive (Java 21+)
public double area(Shape shape) {
    return switch (shape) {
        case Circle c    -> Math.PI * c.radius() * c.radius();
        case Rectangle r -> r.width() * r.height();
        case Triangle t  -> 0.5 * t.base() * t.height();
        // No default needed -- compiler verifies all cases are covered
    };
}

Switch expressions (Java 14+): Switch can now return values and use arrow syntax to eliminate fall-through bugs.

// BAD - traditional switch with fall-through risk
String dayType;
switch (day) {
    case MONDAY:
    case TUESDAY:
    case WEDNESDAY:
    case THURSDAY:
    case FRIDAY:
        dayType = "Weekday";
        break;     // forget this break and you get a bug
    case SATURDAY:
    case SUNDAY:
        dayType = "Weekend";
        break;
    default:
        dayType = "Unknown";
}

// GOOD - switch expression (Java 14+), no fall-through possible
String dayType = switch (day) {
    case MONDAY, TUESDAY, WEDNESDAY, THURSDAY, FRIDAY -> "Weekday";
    case SATURDAY, SUNDAY -> "Weekend";
};

Pattern matching for instanceof (Java 16+): Eliminates the redundant cast after an instanceof check.

// BAD - old way: check instanceof, then cast
if (obj instanceof String) {
    String s = (String) obj;  // redundant cast
    System.out.println(s.length());
}

// GOOD - pattern matching (Java 16+): check and cast in one step
if (obj instanceof String s) {
    System.out.println(s.length());
}

// Works great with negation too
if (!(obj instanceof String s)) {
    return; // early exit
}
// s is in scope here
System.out.println(s.toUpperCase());

14. Streams and Lambdas

Streams and lambdas are powerful tools for functional-style programming in Java. Used well, they make code concise and declarative. Used poorly, they create unreadable chains that are harder to debug than simple loops.

When to use streams:

  • Transforming a collection (map, filter, reduce)
  • Collecting results into a new collection
  • Aggregating data (sum, average, grouping)
  • Finding elements (findFirst, anyMatch)

When NOT to use streams:

  • Simple iteration with side effects (use for-each)
  • When you need to modify local variables (streams require effectively final)
  • When the stream pipeline becomes longer than 5-6 operations (extract to methods)
// BAD - overly complex stream, hard to read and debug
Map> result = orders.stream()
    .filter(o -> o.getStatus() != null && o.getStatus().equals("ACTIVE")
        && o.getTotal() != null && o.getTotal().compareTo(BigDecimal.ZERO) > 0)
    .sorted(Comparator.comparing(Order::getCreatedAt).reversed())
    .map(o -> new OrderDTO(o.getId(), o.getCustomer().getName(),
        o.getItems().stream().map(Item::getName).collect(Collectors.joining(", ")),
        o.getTotal()))
    .collect(Collectors.groupingBy(OrderDTO::customerName));

// GOOD - break it into readable steps
List activeOrders = orders.stream()
    .filter(this::isActiveWithPositiveTotal)
    .sorted(Comparator.comparing(Order::getCreatedAt).reversed())
    .toList();

Map> result = activeOrders.stream()
    .map(this::toOrderDTO)
    .collect(Collectors.groupingBy(OrderDTO::customerName));

// Helper methods make it self-documenting
private boolean isActiveWithPositiveTotal(Order order) {
    return "ACTIVE".equals(order.getStatus())
        && order.getTotal() != null
        && order.getTotal().compareTo(BigDecimal.ZERO) > 0;
}

private OrderDTO toOrderDTO(Order order) {
    String itemNames = order.getItems().stream()
        .map(Item::getName)
        .collect(Collectors.joining(", "));
    return new OrderDTO(order.getId(),
        order.getCustomer().getName(), itemNames, order.getTotal());
}

Prefer method references over lambdas when the lambda simply delegates to an existing method:

// OK - lambda
names.stream().map(name -> name.toUpperCase()).collect(Collectors.toList());
names.stream().filter(name -> name.isEmpty()).count();

// BETTER - method reference (cleaner, less noise)
names.stream().map(String::toUpperCase).collect(Collectors.toList());
names.stream().filter(String::isEmpty).count();

// More method reference examples
users.stream().map(User::getEmail)...                  // instance method
numbers.stream().forEach(System.out::println);          // static method
names.stream().map(String::new)...                      // constructor

Avoid side effects in streams. Streams are designed for functional-style operations. Modifying external state inside a stream is confusing and can cause bugs with parallel streams.

// BAD - side effects: modifying external list inside a stream
List results = new ArrayList<>();
names.stream()
    .filter(n -> n.length() > 3)
    .forEach(n -> results.add(n));  // mutating external state

// GOOD - collect into a new list (no side effects)
List results = names.stream()
    .filter(n -> n.length() > 3)
    .collect(Collectors.toList());

// Or with Java 16+
List results = names.stream()
    .filter(n -> n.length() > 3)
    .toList();



Testing and Quality

Code without tests is legacy code the moment it is written. These practices ensure your code is testable and that your tests are useful.

15. Write Testable Code

Testability is a design quality. If your code is hard to test, that is a sign of poor design. The following principles make code easy to test:

Use dependency injection. Classes that create their own dependencies are impossible to test in isolation.

// BAD - untestable: creates its own dependencies
public class OrderProcessor {
    private final PaymentGateway gateway = new StripePaymentGateway();
    private final EmailService email = new SmtpEmailService();

    public void process(Order order) {
        gateway.charge(order.getTotal());  // hits real Stripe API in tests!
        email.send(order.getEmail(), "Order confirmed");  // sends real emails!
    }
}

// GOOD - testable: dependencies are injected
public class OrderProcessor {
    private final PaymentGateway gateway;
    private final EmailService email;

    public OrderProcessor(PaymentGateway gateway, EmailService email) {
        this.gateway = gateway;
        this.email = email;
    }

    public void process(Order order) {
        gateway.charge(order.getTotal());
        email.send(order.getEmail(), "Order confirmed");
    }
}

// In tests, inject fakes or mocks:
@Test
void shouldChargeAndSendEmail() {
    var fakeGateway = new FakePaymentGateway();
    var fakeEmail = new FakeEmailService();
    var processor = new OrderProcessor(fakeGateway, fakeEmail);

    processor.process(testOrder);

    assertTrue(fakeGateway.wasCharged());
    assertEquals("Order confirmed", fakeEmail.getLastMessage());
}

Keep methods small and focused. A method that does one thing is easy to test. A method that does five things requires a combinatorial explosion of test cases.

Use meaningful test names that describe the scenario and expected behavior:

// BAD - test names that tell you nothing when they fail
@Test void test1() { ... }
@Test void testProcess() { ... }
@Test void testUser() { ... }

// GOOD - test names describe scenario and expected behavior
@Test void withdraw_withSufficientFunds_shouldReduceBalance() { ... }
@Test void withdraw_withInsufficientFunds_shouldThrowException() { ... }
@Test void withdraw_withNegativeAmount_shouldThrowIllegalArgument() { ... }
@Test void findByEmail_withNonexistentEmail_shouldReturnEmpty() { ... }

// ALSO GOOD - Given/When/Then style
@Test void givenInactiveUser_whenLogin_thenAccountLockedExceptionThrown() { ... }

Avoid static methods for business logic. Static methods cannot be overridden, mocked, or injected. They couple your code to a specific implementation.

// BAD - static utility for business logic is untestable in callers
public class PricingUtils {
    public static BigDecimal calculateDiscount(Order order) {
        // Complex business logic tied to a static method
        // Callers cannot replace this in tests
    }
}

// GOOD - instance method on an injectable service
public class PricingService {
    public BigDecimal calculateDiscount(Order order) {
        // Same logic, but now callers can inject a fake
    }
}

// Static methods are fine for pure utility functions with no side effects:
public final class StringUtils {
    public static boolean isBlank(String s) {
        return s == null || s.trim().isEmpty();
    }
}

16. Logging Best Practices

Logging is your primary tool for understanding what your application does in production. Poor logging means you are flying blind when things go wrong.

Use SLF4J as the logging facade. SLF4J lets you swap the underlying implementation (Logback, Log4j2) without changing application code.

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class PaymentService {
    // One logger per class, declared as a static final field
    private static final Logger log = LoggerFactory.getLogger(PaymentService.class);

    public PaymentResult processPayment(PaymentRequest request) {
        log.info("Processing payment for order {}", request.getOrderId());

        try {
            PaymentResult result = gateway.charge(request);
            log.info("Payment successful: orderId={}, transactionId={}",
                request.getOrderId(), result.getTransactionId());
            return result;
        } catch (PaymentDeclinedException e) {
            log.warn("Payment declined for order {}: {}",
                request.getOrderId(), e.getReason());
            throw e;
        } catch (Exception e) {
            log.error("Payment processing failed for order {}",
                request.getOrderId(), e);  // exception as last arg logs stack trace
            throw new PaymentProcessingException("Failed to process payment", e);
        }
    }
}

Use the correct log levels:

Level When to Use Example
ERROR Something failed that requires attention Database connection lost, payment failed
WARN Something unexpected but the system recovered Retry succeeded, fallback used, deprecated API called
INFO Significant business events User registered, order placed, payment processed
DEBUG Detailed flow information for troubleshooting Method entry/exit, intermediate values
TRACE Very detailed debugging (rarely used) Loop iterations, full request/response bodies

Use parameterized logging. Never concatenate strings in log statements. With parameterized logging, the string is only built if the log level is enabled.

// BAD - string concatenation happens even if DEBUG is disabled
log.debug("Processing user " + user.getName() + " with id " + user.getId());

// GOOD - parameterized: {} placeholders are only resolved if level is enabled
log.debug("Processing user {} with id {}", user.getName(), user.getId());

// BAD - checking log level AND using concatenation (defeats the purpose)
if (log.isDebugEnabled()) {
    log.debug("Processing user " + user.getName());
}

// GOOD - parameterized logging handles the check internally
log.debug("Processing user {}", user.getName());

What to log:

  • Application startup and shutdown events
  • Authentication and authorization events (login, logout, access denied)
  • Business transactions (order placed, payment processed)
  • Integration points (API calls, database queries, message queue operations)
  • Errors and exceptions with full context (IDs, parameters)

What NOT to log:

  • Passwords, tokens, credit card numbers, or any sensitive data
  • Full request/response bodies at INFO level (use DEBUG or TRACE)
  • Every loop iteration (use aggregate logging instead)



Code Organization

How you organize your code determines how quickly developers can find what they need, understand the system architecture, and make changes without breaking unrelated features.

17. Project Structure

Package by feature, not by layer. The traditional approach of packaging by layer (controllers, services, repositories) scatters related code across the project. Packaging by feature keeps everything related together.

// BAD - package by layer (a change to "orders" touches 4 packages)
com.example.app
├── controller/
│   ├── OrderController.java
│   ├── UserController.java
│   └── ProductController.java
├── service/
│   ├── OrderService.java
│   ├── UserService.java
│   └── ProductService.java
├── repository/
│   ├── OrderRepository.java
│   ├── UserRepository.java
│   └── ProductRepository.java
└── model/
    ├── Order.java
    ├── User.java
    └── Product.java

// GOOD - package by feature (everything about "orders" is together)
com.example.app
├── order/
│   ├── OrderController.java
│   ├── OrderService.java
│   ├── OrderRepository.java
│   ├── Order.java
│   └── OrderDTO.java
├── user/
│   ├── UserController.java
│   ├── UserService.java
│   ├── UserRepository.java
│   └── User.java
└── product/
    ├── ProductController.java
    ├── ProductService.java
    ├── ProductRepository.java
    └── Product.java

Benefits of packaging by feature:

  • Higher cohesion -- related code lives together
  • Easier to understand a feature by looking at one package
  • Easier to extract a feature into a separate module or microservice
  • Package-private visibility actually works (classes can be package-private)

18. Class Design

Keep classes focused (Single Responsibility Principle). A class should have one, and only one, reason to change. If you find yourself describing a class with "and" ("this class handles orders and sends emails and generates reports"), it has too many responsibilities.

Keep classes small. A class with 2000 lines is a "God class" that tries to do everything. Aim for classes under 200-300 lines. If a class is growing beyond that, look for responsibilities to extract.

Use the Builder pattern when a constructor has many parameters (more than 3-4). Builders are self-documenting and prevent parameter-ordering bugs.

// BAD - constructor with many parameters (which is email? which is phone?)
User user = new User("Alice", "Smith", "alice@example.com",
    "555-1234", "123 Main St", "Springfield", "IL", "62701", true, false);

// GOOD - Builder pattern: every parameter is labeled
User user = User.builder()
    .firstName("Alice")
    .lastName("Smith")
    .email("alice@example.com")
    .phone("555-1234")
    .address(Address.builder()
        .street("123 Main St")
        .city("Springfield")
        .state("IL")
        .zipCode("62701")
        .build())
    .isActive(true)
    .isAdmin(false)
    .build();

Here is a clean Builder implementation:

public class User {
    private final String firstName;
    private final String lastName;
    private final String email;

    private User(Builder builder) {
        this.firstName = Objects.requireNonNull(builder.firstName, "firstName required");
        this.lastName = Objects.requireNonNull(builder.lastName, "lastName required");
        this.email = Objects.requireNonNull(builder.email, "email required");
    }

    public static Builder builder() {
        return new Builder();
    }

    // Getters...
    public String getFirstName() { return firstName; }
    public String getLastName() { return lastName; }
    public String getEmail() { return email; }

    public static class Builder {
        private String firstName;
        private String lastName;
        private String email;

        public Builder firstName(String firstName) {
            this.firstName = firstName;
            return this;
        }

        public Builder lastName(String lastName) {
            this.lastName = lastName;
            return this;
        }

        public Builder email(String email) {
            this.email = email;
            return this;
        }

        public User build() {
            return new User(this);
        }
    }
}



Security

Security is not something you bolt on at the end. It must be part of how you write every line of code. These practices protect your application from the most common vulnerabilities.

19. Security Best Practices

Always validate input. Never trust data from external sources -- user input, API responses, file uploads, URL parameters. Validate, sanitize, and constrain before processing.

// BAD - no input validation
public void createUser(String name, String email, int age) {
    // Directly uses untrusted input
    User user = new User(name, email, age);
    repository.save(user);
}

// GOOD - validate everything
public void createUser(String name, String email, int age) {
    if (name == null || name.isBlank()) {
        throw new IllegalArgumentException("Name must not be blank");
    }
    if (name.length() > 100) {
        throw new IllegalArgumentException("Name must not exceed 100 characters");
    }
    if (email == null || !EMAIL_PATTERN.matcher(email).matches()) {
        throw new IllegalArgumentException("Invalid email format");
    }
    if (age < 0 || age > 150) {
        throw new IllegalArgumentException("Age must be between 0 and 150");
    }
    User user = new User(name.trim(), email.toLowerCase().trim(), age);
    repository.save(user);
}

private static final Pattern EMAIL_PATTERN =
    Pattern.compile("^[A-Za-z0-9+_.-]+@[A-Za-z0-9.-]+\\.[A-Za-z]{2,}$");

Use parameterized queries. SQL injection is one of the most dangerous and preventable vulnerabilities. Never build SQL strings by concatenating user input.

// BAD - SQL injection vulnerability!
// If username is: ' OR '1'='1' -- , this returns ALL users
public User findUser(String username) {
    String sql = "SELECT * FROM users WHERE username = '" + username + "'";
    return jdbcTemplate.queryForObject(sql, userRowMapper);
}

// GOOD - parameterized query: input is treated as DATA, never as SQL
public User findUser(String username) {
    String sql = "SELECT * FROM users WHERE username = ?";
    return jdbcTemplate.queryForObject(sql, userRowMapper, username);
}

// GOOD - using JPA named parameters
@Query("SELECT u FROM User u WHERE u.username = :username")
Optional findByUsername(@Param("username") String username);

Never hardcode secrets. Passwords, API keys, database credentials, and tokens must never appear in source code. Use environment variables, secret managers, or configuration files that are excluded from version control.

// BAD - secrets in source code (visible in git history forever)
public class DatabaseConfig {
    private static final String DB_PASSWORD = "MyS3cretP@ssw0rd";
    private static final String API_KEY = "sk-1234567890abcdef";
}

// GOOD - read from environment variables
public class DatabaseConfig {
    private final String dbPassword = System.getenv("DB_PASSWORD");
    private final String apiKey = System.getenv("API_KEY");

    public DatabaseConfig() {
        if (dbPassword == null || dbPassword.isBlank()) {
            throw new IllegalStateException(
                "DB_PASSWORD environment variable is not set");
        }
    }
}

// GOOD - Spring Boot: use application.yml with environment variable references
// spring:
//   datasource:
//     password: ${DB_PASSWORD}
// API keys read from a vault or secrets manager in production

Apply the principle of least privilege. Give classes, methods, and fields the minimum visibility they need. Default to private and only widen access when there is a clear reason.

// BAD - everything is public (exposes internal implementation)
public class UserService {
    public List userCache = new ArrayList<>();  // internal state exposed
    public Connection dbConnection;                    // anyone can access this

    public String hashPassword(String password) { ... }  // internal method exposed
    public void updateCache(User user) { ... }           // internal method exposed
}

// GOOD - minimal visibility
public class UserService {
    private final List userCache = new ArrayList<>();
    private final DataSource dataSource;

    public UserService(DataSource dataSource) {
        this.dataSource = dataSource;
    }

    // Only public methods that form the API contract
    public User findById(long id) { ... }
    public void register(User user) { ... }

    // Internal helpers are private
    private String hashPassword(String password) { ... }
    private void updateCache(User user) { ... }
}



20. Summary Quick Reference Table

Here is every best practice from this tutorial in one scannable table. Bookmark this page and use it as a checklist during code reviews.

# Best Practice Do NOT DO
1 Naming Conventions int d;, String addr;, void calc(); int daysSinceCreation;, String address;, void calculateTotal();
2 Code Formatting No braces on if/for, inconsistent indentation Always use braces, 4-space indent, IDE auto-format
3 Comments // increment i by 1, comments that repeat code Comment the why, Javadoc for public APIs
4 SOLID Principles God classes with 10 responsibilities One class, one responsibility, depend on abstractions
5 Composition over Inheritance class UserService extends Logger class UserService { private Logger logger; }
6 Program to Interfaces ArrayList<String> names = new ArrayList<>(); List<String> names = new ArrayList<>();
7 Immutability Public mutable fields, expose internal lists private final fields, defensive copies, List.copyOf()
8 Exception Handling catch (Exception e) { } Catch specific exceptions, try-with-resources, fail fast
9 Null Safety Return null for "not found" Return Optional, empty collections, Objects.requireNonNull()
10 String Handling result += line in a loop StringBuilder, String.join(), text blocks
11 Collections LinkedList for random access, no pre-sizing Right collection for the job, pre-size when known
12 Resource Management Manual close in finally blocks try-with-resources for all AutoCloseable objects
13 Modern Java Features 50-line boilerplate POJOs Records, sealed classes, pattern matching, switch expressions
14 Streams and Lambdas 10-operation chain, side effects in streams Break chains into methods, use method references, no side effects
15 Testable Code new StripeGateway() inside the class Constructor injection, small methods, meaningful test names
16 Logging log.debug("user " + name), inconsistent levels SLF4J with {} placeholders, correct log levels
17 Project Structure Package by layer (controller/service/repo) Package by feature (order/user/product)
18 Class Design 2000-line God class, 10-parameter constructors Small focused classes, Builder pattern
19 Security String concatenation in SQL, hardcoded passwords Parameterized queries, environment variables, input validation

Final Thoughts

Best practices are not rules carved in stone -- they are guidelines refined through decades of collective experience. There will be times when breaking a guideline makes sense, but you should always be able to articulate why you are breaking it.

Start by adopting these practices one at a time. Pick the ones that address your biggest pain points first. As they become habits, your code will become easier to read, safer to change, and more enjoyable to work with -- for you and for everyone on your team.

The best code is code that future developers (including future you) can understand, modify, and extend without fear.

August 12, 2018

Intellij Hot Keys

Reformat Code

For Mac

⌥ ⌘ L

command + option + l

View Structure/Outline of Class

command + 7

 

 

 

June 7, 2018

Eclipse Hot Keys




1. Why Learn IDE Shortcuts?

The difference between a junior developer and a senior developer is often not just knowledge of design patterns or architecture — it is how fast they can translate ideas into code. IDE shortcuts are one of the biggest productivity multipliers available to you.

Consider this: every time you reach for the mouse to navigate a menu, find a file, or refactor a variable name, you lose 2-5 seconds. That does not sound like much, but a developer performs these actions hundreds of times per day. Over a year, those seconds add up to weeks of lost productivity.

Here is what changes when you master IDE shortcuts:

  • Navigation becomes instant — jump to any class, method, or line in milliseconds instead of scrolling or searching
  • Refactoring becomes fearless — rename a variable across 200 files with a single shortcut instead of find-and-replace
  • Code generation is automatic — generate constructors, getters/setters, equals/hashCode in seconds
  • Debugging is faster — set breakpoints, step through code, and evaluate expressions without touching the mouse
  • You stay in flow state — context switching between keyboard and mouse breaks concentration

This tutorial covers shortcuts for the two most popular Java IDEs: Eclipse and IntelliJ IDEA. The shortcuts shown use Windows/Linux keybindings. On macOS, replace Ctrl with Cmd and Alt with Option unless noted otherwise.

Tip: Do not try to memorize all shortcuts at once. Pick 3-5 from this list, use them exclusively for a week until they become muscle memory, then add 3-5 more. Within a few months, you will be dramatically faster.



2. Essential Navigation Shortcuts

Navigation shortcuts let you jump to any part of the codebase instantly. These are the shortcuts you will use most frequently — master them first.

Action Eclipse IntelliJ IDEA Notes
Go to Class Ctrl + Shift + T Ctrl + N Type any part of the class name. Supports CamelCase (e.g., “USC” finds UserServiceController)
Go to File / Resource Ctrl + Shift + R Ctrl + Shift + N Opens any file — .xml, .properties, .html, .java, etc.
Go to Line Number Ctrl + L Ctrl + G Jump to a specific line number in the current file
Go to Symbol / Method Ctrl + O Ctrl + F12 Shows all methods and fields in the current class. Start typing to filter.
Go to Declaration F3 or Ctrl + Click Ctrl + B or Ctrl + Click Jump to where a variable, method, or class is defined
Go to Implementation Ctrl + T Ctrl + Alt + B From an interface method, jump to its implementations
Navigate Back Alt + Left Ctrl + Alt + Left Go back to where you were before (like browser back button)
Navigate Forward Alt + Right Ctrl + Alt + Right Undo a “navigate back”
Recent Files Ctrl + E Ctrl + E Shows list of recently opened files (same in both IDEs)
Switch Between Tabs Ctrl + Page Up / Down Alt + Left / Right Cycle through open editor tabs
Show Type Hierarchy F4 Ctrl + H See the inheritance tree of a class
Search Everywhere N/A Double Shift IntelliJ-only: search classes, files, actions, settings — everything
Quick Open Type Ctrl + Shift + T Ctrl + N Open any class by name, supports wildcard (*) patterns

Pro tip: IntelliJ’s Double Shift (Search Everywhere) is arguably the single most powerful shortcut. It searches classes, files, methods, settings, actions, and even Git branches. If you only learn one IntelliJ shortcut, make it this one.

Pro tip: Eclipse’s Ctrl + Shift + T supports CamelCase shortcuts. Instead of typing “ApplicationContextConfiguration”, type “ACC” and it will find the class. This works in IntelliJ too with Ctrl + N.



3. Code Editing Shortcuts

These shortcuts make writing and rearranging code faster. Once you learn them, you will never manually copy/paste to move a line or type a comment prefix again.

Action Eclipse IntelliJ IDEA Notes
Duplicate Line Ctrl + Alt + Down Ctrl + D Duplicates the current line or selected block below
Delete Line Ctrl + D Ctrl + Y Deletes the entire current line (careful: same key, different action!)
Move Line Up Alt + Up Alt + Shift + Up Moves the current line or selected block up
Move Line Down Alt + Down Alt + Shift + Down Moves the current line or selected block down
Comment Line Ctrl + / Ctrl + / Toggle // comment on current line or selection (same in both)
Block Comment Ctrl + Shift + / Ctrl + Shift + / Wrap selection in /* */ block comment
Auto-Format Code Ctrl + Shift + F Ctrl + Alt + L Reformats the entire file or selected code according to style rules
Auto-Import Ctrl + Shift + O Ctrl + Alt + O Eclipse: organizes imports (adds missing, removes unused). IntelliJ: removes unused (auto-import adds on the fly)
Quick Fix / Suggestions Ctrl + 1 Alt + Enter Shows context actions: create method, add import, fix error, etc.
Complete Code Ctrl + Space Ctrl + Space Autocomplete methods, variables, types (same in both)
Smart Complete N/A Ctrl + Shift + Space IntelliJ-only: context-aware completion (filters by expected type)
Surround With Alt + Shift + Z Ctrl + Alt + T Wrap selected code in try/catch, if/else, for loop, etc.
Select Word Double-click Ctrl + W IntelliJ: expands selection progressively (word > statement > block > method > class)
Shrink Selection N/A Ctrl + Shift + W IntelliJ-only: reverse of Ctrl + W
Undo Ctrl + Z Ctrl + Z Standard undo (same in both)
Redo Ctrl + Y Ctrl + Shift + Z Note: Ctrl + Y in IntelliJ deletes a line, NOT redo!

Warning: Notice that Ctrl + D does completely different things in Eclipse (delete line) versus IntelliJ (duplicate line). Similarly, Ctrl + Y is redo in Eclipse but delete line in IntelliJ. If you switch between IDEs, this will trip you up. IntelliJ offers an “Eclipse keymap” option (Settings > Keymap) that remaps IntelliJ shortcuts to match Eclipse, which can ease the transition.



4. Code Generation

Code generation shortcuts automatically create boilerplate code that would take minutes to write by hand. These are especially useful for Java, which requires a lot of boilerplate (constructors, getters/setters, equals, hashCode, toString).

Action Eclipse IntelliJ IDEA Notes
Generate Menu Alt + Shift + S Alt + Insert Opens the generation menu with all options below
Generate Constructor Alt + Shift + S then O Alt + Insert > Constructor Select which fields to include in the constructor
Generate Getters/Setters Alt + Shift + S then R Alt + Insert > Getter and Setter Select fields to generate getters/setters for
Generate toString() Alt + Shift + S then S Alt + Insert > toString() Creates a toString() method including selected fields
Generate equals() / hashCode() Alt + Shift + S then H Alt + Insert > equals() and hashCode() Creates both methods based on selected fields
Override Methods Alt + Shift + S then V Ctrl + O Override a method from a parent class
Implement Interface Methods Alt + Shift + S then V Ctrl + I Implement abstract methods from an interface
Generate Delegate Methods Alt + Shift + S then E Alt + Insert > Delegate Methods Delegate to a field (composition pattern)

Here is an example of how quickly you can build a complete Java class using generation shortcuts:

// Step 1: Type the class with fields (the only part you type manually)
public class Employee {
    private Long id;
    private String firstName;
    private String lastName;
    private String email;
    private double salary;
}

// Step 2: Place cursor inside the class body
// Press Alt + Insert (IntelliJ) or Alt + Shift + S (Eclipse)
// Select "Constructor" > choose all fields > OK
// Result: Constructor generated automatically

// Step 3: Press Alt + Insert again > select "Getter and Setter" > select all fields
// Result: 10 methods generated (5 getters + 5 setters)

// Step 4: Press Alt + Insert > "toString()" > select all fields
// Result: toString() generated

// Step 5: Press Alt + Insert > "equals() and hashCode()" > select id and email
// Result: Both methods generated

// Total time: ~15 seconds for a complete domain class
// Without shortcuts: 5-10 minutes of typing

Modern alternative: If you are on Java 16+, consider using records instead. A record auto-generates the constructor, getters, equals, hashCode, and toString in a single line:

// Java 16+ record -- replaces all the generated boilerplate above
public record Employee(Long id, String firstName, String lastName, String email, double salary) {}

// Constructor, getters (id(), firstName(), etc.), equals(), hashCode(), toString()
// are ALL auto-generated by the compiler



5. Refactoring Shortcuts

Refactoring shortcuts are what separate IDE users from text editor users. These shortcuts let you safely restructure code without manually finding and replacing text across files — the IDE understands Java syntax and only changes the correct references.

Action Eclipse IntelliJ IDEA Notes
Rename Alt + Shift + R Shift + F6 Renames a variable, method, class, or package across ALL references in the project. The most important refactoring shortcut.
Extract Variable Alt + Shift + L Ctrl + Alt + V Extracts a selected expression into a new local variable
Extract Constant N/A (use menu) Ctrl + Alt + C Extracts a value into a static final constant
Extract Field Alt + Shift + F Ctrl + Alt + F Extracts a value into an instance field
Extract Method Alt + Shift + M Ctrl + Alt + M Extracts selected code into a new method. The IDE figures out the parameters and return type.
Extract Parameter N/A (use menu) Ctrl + Alt + P Converts a local variable or expression into a method parameter
Inline Alt + Shift + I Ctrl + Alt + N Opposite of extract: replaces a variable/method with its value/body
Move Class Alt + Shift + V F6 Move a class to a different package (updates all imports)
Change Method Signature Alt + Shift + C Ctrl + F6 Add/remove/reorder parameters, change return type
Refactoring Menu Alt + Shift + T Ctrl + Alt + Shift + T Shows all available refactoring options for the selected code

Rename is the refactoring you will use most often. It is drastically safer than find-and-replace because the IDE understands Java:

// Before renaming: variable "name" is used in multiple places
public class UserService {

    public User findUser(String name) {           // parameter "name"
        log.info("Searching for: " + name);       // reference to "name"
        User user = repository.findByName(name);  // reference to "name"
        if (user == null) {
            throw new UserNotFoundException("User not found: " + name);
        }
        return user;
    }
}

// Place cursor on "name" and press Shift + F6 (IntelliJ) or Alt + Shift + R (Eclipse)
// Type "username" and press Enter
// ALL references update simultaneously -- but "findByName" (different variable) is NOT changed

// After renaming:
public class UserService {

    public User findUser(String username) {           // renamed
        log.info("Searching for: " + username);       // renamed
        User user = repository.findByName(username);  // renamed (the argument)
        if (user == null) {
            throw new UserNotFoundException("User not found: " + username);
        }
        return user;
    }
}

// The method name "findByName" was correctly left unchanged because
// it is a different identifier. Find-and-replace would have broken it.

Extract Method is the second most important refactoring. It takes a block of code, moves it into a new method, and replaces the original code with a call to the new method:

// Before: a long method doing too many things
public void processOrder(Order order) {
    // Select these lines, then press Ctrl + Alt + M (IntelliJ) or Alt + Shift + M (Eclipse)
    double subtotal = 0;
    for (OrderItem item : order.getItems()) {
        subtotal += item.getPrice() * item.getQuantity();
    }
    double tax = subtotal * 0.08;
    double total = subtotal + tax;
    order.setTotal(total);
    // End selection

    sendConfirmationEmail(order);
    updateInventory(order);
}

// After: the IDE extracts it into a new method automatically
public void processOrder(Order order) {
    calculateTotal(order);    // extracted method call
    sendConfirmationEmail(order);
    updateInventory(order);
}

// The IDE generated this method with the correct parameter and logic
private void calculateTotal(Order order) {
    double subtotal = 0;
    for (OrderItem item : order.getItems()) {
        subtotal += item.getPrice() * item.getQuantity();
    }
    double tax = subtotal * 0.08;
    double total = subtotal + tax;
    order.setTotal(total);
}



6. Debugging Shortcuts

Debugging shortcuts let you step through code, inspect variables, and find bugs without constantly clicking buttons in the debug toolbar. These are essential for efficient debugging sessions.

Action Eclipse IntelliJ IDEA Notes
Toggle Breakpoint Ctrl + Shift + B Ctrl + F8 Set or remove a breakpoint on the current line
Run in Debug Mode F11 Shift + F9 Start the application in debug mode
Run (No Debug) Ctrl + F11 Shift + F10 Run without debugging
Step Over F6 F8 Execute the current line and move to the next (skip method internals)
Step Into F5 F7 Enter into the method call on the current line
Step Out F7 Shift + F8 Finish the current method and return to the caller
Resume F8 F9 Continue execution until the next breakpoint
Run to Cursor Ctrl + R Alt + F9 Run until execution reaches the line where your cursor is (temporary breakpoint)
Evaluate Expression Ctrl + Shift + I Alt + F8 Evaluate any Java expression during debugging (inspect variables, call methods)
View Breakpoints Breakpoints View Ctrl + Shift + F8 See all breakpoints, add conditions, enable/disable
Stop Ctrl + F2 Ctrl + F2 Terminate the running application (same in both)
Smart Step Into N/A Shift + F7 IntelliJ-only: when a line has multiple method calls, choose which one to step into

Pro tip: In IntelliJ, you can set conditional breakpoints. Right-click on a breakpoint and enter a condition like user.getId() == 42. The breakpoint will only trigger when the condition is true. This is incredibly useful when debugging loops or processing lists of data.

Pro tip: In both IDEs, you can change variable values during debugging. In the Variables panel, right-click a variable and select “Set Value.” This lets you test different scenarios without restarting the application.



7. Search Shortcuts

Search shortcuts help you find anything in your codebase. Whether you are looking for a method call, a string literal, or all usages of a class, these shortcuts get you there instantly.

Action Eclipse IntelliJ IDEA Notes
Find in Current File Ctrl + F Ctrl + F Basic find in the current editor (same in both)
Find and Replace Ctrl + H Ctrl + R Find and replace in current file. Supports regex.
Find in Path / Project Ctrl + H (Search tab) Ctrl + Shift + F Search across entire project, with file filters and regex
Replace in Path / Project Ctrl + H (Search tab) Ctrl + Shift + R Find and replace across entire project
Find Usages Ctrl + Shift + G Alt + F7 Find all places where a class, method, or variable is used
Highlight Usages Ctrl + Shift + G Ctrl + Shift + F7 IntelliJ highlights all usages in the current file
Find Next Ctrl + K F3 or Enter Jump to next search result
Find Previous Ctrl + Shift + K Shift + F3 Jump to previous search result
Search Everywhere N/A Double Shift IntelliJ-only: searches everything — classes, files, actions, settings, git branches
Find Action / Command Ctrl + 3 Ctrl + Shift + A Search for any IDE action by name (useful when you forget a shortcut)

Find Usages (Alt + F7 in IntelliJ / Ctrl + Shift + G in Eclipse) is one of the most valuable search tools. Before changing or deleting a method, use Find Usages to see every place it is called. This prevents you from breaking code you did not know about.

Pro tip: When you forget a shortcut, use Find Action (Ctrl + Shift + A in IntelliJ / Ctrl + 3 in Eclipse). Type what you want to do (e.g., “extract method”) and the IDE will show you the action along with its keyboard shortcut.



8. Version Control Shortcuts

Both Eclipse (with EGit) and IntelliJ IDEA have built-in Git support. These shortcuts let you commit, push, pull, and review changes without leaving the IDE or opening a terminal.

Action Eclipse (EGit) IntelliJ IDEA Notes
Commit Ctrl + # (Team menu) Ctrl + K Open the commit dialog with staged changes
Push Team > Push Ctrl + Shift + K Push committed changes to remote
Pull / Update Team > Pull Ctrl + T Pull latest changes from remote
Show Diff Compare With > HEAD Ctrl + D (in commit dialog) See what changed in a file compared to the last commit
Show History / Log Team > Show in History Alt + 9 View the Git log for a file or the entire project
Annotate / Blame Team > Show Annotations Right-click gutter > Annotate See who wrote each line and when (git blame)
VCS Operations Menu N/A Alt + ` (backtick) IntelliJ-only: quick access to all VCS operations
Rollback / Revert Replace With > HEAD Ctrl + Alt + Z Revert selected file changes to the last committed version
Create Branch Team > Switch To > New Branch Git widget (bottom-right) or VCS menu Both IDEs let you create and switch branches from the IDE

Pro tip: IntelliJ’s commit dialog (Ctrl + K) lets you review every changed file, see the diff, write a commit message, and amend the previous commit — all in one place. You can also run code analysis before committing to catch issues.



9. Live Templates / Code Templates

Live templates (IntelliJ) and code templates (Eclipse) let you type a short abbreviation and expand it into a complete code snippet. Both IDEs come with built-in templates, and you can create your own.

Built-in Templates

Abbreviation Expansion Eclipse IntelliJ
sout System.out.println(); sysout + Ctrl + Space sout + Tab
souf System.out.printf(); N/A (manual) souf + Tab
soutv System.out.println("var = " + var); N/A soutv + Tab
main public static void main(String[] args) { } main + Ctrl + Space psvm + Tab
fori for (int i = 0; i < ; i++) { } for + Ctrl + Space fori + Tab
foreach for (Type item : collection) { } foreach + Ctrl + Space iter + Tab
ifn if (var == null) { } N/A ifn + Tab
inn if (var != null) { } N/A inn + Tab
thr throw new N/A thr + Tab
St String N/A St + Tab
psf public static final N/A psf + Tab
psfs public static final String N/A psfs + Tab

IntelliJ also supports postfix completion, which is uniquely powerful. You type an expression first, then add a template suffix:

// Postfix completion (IntelliJ only) -- type the expression FIRST, then the template

// .var -- creates a variable from an expression
new ArrayList().var    // press Tab -->  List strings = new ArrayList<>();

// .if -- wraps in an if statement
user != null.if                // press Tab -->  if (user != null) { }

// .not -- negates a boolean
isValid.not                    // press Tab -->  !isValid

// .for -- creates a for-each loop
users.for                      // press Tab -->  for (User user : users) { }

// .fori -- creates an indexed for loop
users.fori                     // press Tab -->  for (int i = 0; i < users.size(); i++) { }

// .null -- null check
result.null                    // press Tab -->  if (result == null) { }

// .nn -- not-null check
result.nn                      // press Tab -->  if (result != null) { }

// .try -- wraps in try-catch
inputStream.read().try         // press Tab -->  try { inputStream.read(); } catch (Exception e) { }

// .return -- adds return statement
calculateTotal().return        // press Tab -->  return calculateTotal();

// .sout -- wraps in System.out.println
result.sout                    // press Tab -->  System.out.println(result);

Creating Custom Templates

IntelliJ: Go to Settings > Editor > Live Templates. Click the + button, define an abbreviation, write the template text, and set the context (Java, etc.).

Eclipse: Go to Window > Preferences > Java > Editor > Templates. Click "New", define a name, and write the template pattern.

Here are some useful custom templates you should create:

// Custom template: "log" -- creates a logger declaration
// Abbreviation: log
// Template:
private static final Logger log = LoggerFactory.getLogger($CLASS_NAME$.class);
// Set $CLASS_NAME$ to className() expression in IntelliJ

// Custom template: "todo" -- creates a TODO comment with date
// Abbreviation: todo
// Template:
// TODO ($USER$ - $DATE$): $END$
// Set $USER$ to user(), $DATE$ to date(), $END$ is where cursor lands

// Custom template: "test" -- creates a JUnit test method
// Abbreviation: test
// Template:
@Test
void should$NAME$() {
    // Given
    $END$

    // When

    // Then
}

// Custom template: "builder" -- creates a builder pattern skeleton
// Abbreviation: builder
// Template:
public static class Builder {
    public Builder $FIELD_NAME$($TYPE$ $PARAM$) {
        this.$FIELD_NAME$ = $PARAM$;
        return this;
    }

    public $CLASS_NAME$ build() {
        return new $CLASS_NAME$(this);
    }
}

Eclipse template variables: Eclipse uses ${cursor} for cursor position, ${enclosing_type} for the class name, ${word_selection} for selected text, and ${date} for the current date.

IntelliJ template variables: IntelliJ uses $END$ for cursor position and supports expressions like className(), date(), user(), suggestVariableName(), and completeSmart() for intelligent auto-fill.



10. Customizing Shortcuts

Both IDEs allow you to customize any keyboard shortcut. This is useful when you want to match your muscle memory from another editor, fix conflicts with your OS, or assign shortcuts to frequently-used actions that have no default binding.

Customizing in Eclipse

  1. Go to Window > Preferences > General > Keys
  2. Use the filter box to search for any command by name
  3. Click on the command you want to change
  4. Click in the "Binding" field and press your desired key combination
  5. Set the "When" context (e.g., "Editing Java Source" or "In Windows")
  6. Click "Apply and Close"

Customizing in IntelliJ

  1. Go to Settings > Keymap
  2. Search for any action in the search box
  3. Right-click the action and select "Add Keyboard Shortcut"
  4. Press your desired key combination and click OK
  5. If there is a conflict, IntelliJ will warn you and let you reassign

Pre-built Keymaps

IntelliJ provides pre-built keymaps that match other editors:

Keymap Matches When to Use
IntelliJ IDEA Classic IntelliJ defaults New IntelliJ users or those who learned IntelliJ first
Eclipse Eclipse keybindings Developers transitioning from Eclipse to IntelliJ
VS Code Visual Studio Code bindings Developers coming from VS Code
NetBeans NetBeans keybindings Developers transitioning from NetBeans
macOS macOS conventions (Cmd-based) Mac users (default on macOS IntelliJ)

Recommendation: If you are switching from Eclipse to IntelliJ, start with the Eclipse keymap to maintain productivity. Over time, gradually learn the IntelliJ-specific shortcuts (like Double Shift, postfix completion, and Ctrl + W for expanding selection) since they have no Eclipse equivalent and are genuinely more powerful.



11. Top 20 Must-Know Shortcuts

If you take nothing else from this tutorial, learn these 20 shortcuts. They cover the actions you perform most frequently and will give you the biggest productivity boost. The shortcuts are ordered by how often a typical Java developer uses them.

# Action Eclipse IntelliJ IDEA Why It Matters
1 Autocomplete Ctrl + Space Ctrl + Space Fastest way to write code -- type 2-3 chars and let the IDE finish
2 Quick Fix Ctrl + 1 Alt + Enter Fixes errors, adds imports, creates methods, suggests improvements
3 Go to Class Ctrl + Shift + T Ctrl + N Jump to any class instantly instead of browsing the project tree
4 Go to Declaration F3 Ctrl + B Jump to where something is defined -- essential for code exploration
5 Rename Alt + Shift + R Shift + F6 Safely rename across the entire project in one action
6 Find in Project Ctrl + H Ctrl + Shift + F Search for any text, regex, or pattern across all files
7 Format Code Ctrl + Shift + F Ctrl + Alt + L Auto-format to maintain consistent code style
8 Organize Imports Ctrl + Shift + O Ctrl + Alt + O Add missing imports and remove unused ones in one keystroke
9 Comment Line Ctrl + / Ctrl + / Instantly comment/uncomment code for debugging or testing
10 Duplicate Line Ctrl + Alt + Down Ctrl + D Copy a line without cut/paste -- great for repetitive code
11 Delete Line Ctrl + D Ctrl + Y Remove an entire line without selecting it first
12 Move Line Alt + Up/Down Alt + Shift + Up/Down Rearrange code without cut/paste
13 Find Usages Ctrl + Shift + G Alt + F7 See everywhere a method/variable/class is used before changing it
14 Generate Code Alt + Shift + S Alt + Insert Generate constructors, getters, setters, toString, equals, hashCode
15 Extract Method Alt + Shift + M Ctrl + Alt + M Break long methods into smaller, readable pieces
16 Surround With Alt + Shift + Z Ctrl + Alt + T Wrap code in try/catch, if/else, loop without manual typing
17 Navigate Back Alt + Left Ctrl + Alt + Left Go back to where you were before (like browser back button)
18 Toggle Breakpoint Ctrl + Shift + B Ctrl + F8 Set/remove breakpoints without clicking in the margin
19 Step Over (Debug) F6 F8 Execute one line at a time during debugging
20 Search Everywhere Ctrl + 3 Double Shift Find anything -- the ultimate "I do not know where this is" shortcut

Learning Strategy

Here is a practical plan for learning these shortcuts:

  • Week 1: Shortcuts 1-5 (Autocomplete, Quick Fix, Go to Class, Go to Declaration, Rename). These five alone will transform your productivity.
  • Week 2: Shortcuts 6-10 (Find in Project, Format, Imports, Comment, Duplicate). These handle code formatting and search.
  • Week 3: Shortcuts 11-15 (Delete Line, Move Line, Find Usages, Generate, Extract Method). These cover editing and refactoring.
  • Week 4: Shortcuts 16-20 (Surround, Navigate, Breakpoint, Step Over, Search Everywhere). These round out debugging and navigation.

Print this table and keep it next to your monitor. Every time you reach for the mouse to do one of these actions, stop, look at the table, and use the shortcut instead. Within a month, you will not need the table anymore.

Bonus: IntelliJ Productivity Guide

IntelliJ IDEA has a built-in feature that tracks your shortcut usage. Go to Help > My Productivity (or Help > Productivity Guide in older versions). It shows you which shortcuts you use, which features save you the most time, and which actions you still perform the slow way. Use this to identify your next shortcuts to learn.

Bonus: Eclipse Key Assist

In Eclipse, press and hold Ctrl + Shift + L to see a popup of all available shortcuts. Press it again to open the Keys preferences where you can customize any shortcut. This is useful when you know what you want to do but cannot remember the key combination.

March 18, 2018

System Basics

What are the basics of a system?

The picture below illustrates more less of what a production system made of.

Web browser and mobile app are clients of our system. 

Load Balancer

A load balancer evenly distributes incoming traffic among web servers that are defined in a load-balanced set. Users connect to the public IP of the load balancer directly. With this setup, web servers are unreachable directly by clients anymore. For better security, private IPs are used for communication between servers. A private IP is an IP address reachable only between servers in the same network; however, it is unreachable over the internet. The load balancer communicates with web servers through private IPs.

 

By having more than one server, we successfully solved no failover issue and improved the availability of the web tier. Details are explained below:

  • If server 1 goes offline, all the traffic will be routed to server 2. This prevents the website from going offline. We will also add a new healthy web server to the server pool to balance the load.
  • If the website traffic grows rapidly, and two servers are not enough to handle the traffic, the load balancer can handle this problem gracefully. You only need to add more servers to the web server pool, and the load balancer automatically starts to send requests to them.

Database 
Database job is to store and retreive data. You can choose between a traditional relational database and a non-relational database. Sometimes this is not an easy decision as it depends on the requirements on your system. For most developers, relational databases are the best option because they have been around for over 40 years and historically, they have worked well. However, if relational databases are not suitable for your specific use cases, it is critical to explore beyond relational databases. Non-relational databases might be the right choice if:

  • Your application requires super-low latency.
  • Your data are unstructured, or you do not have any relational data.
  • You only need to serialize and deserialize data (JSON, XML, YAML, etc.).
  • You need to store a massive amount of data.

Database vertical scaling vs horizontal scaling

Vertical scaling, referred to as “scale up”, means the process of adding more power (CPU, RAM, etc.) to your servers. Horizontal scaling, referred to as “scale-out”, allows you to scale by adding more servers into your pool of resources.

When traffic is low, vertical scaling is a great option, and the simplicity of vertical scaling is its main advantage. Unfortunately, it comes with serious limitations.

  • Vertical scaling has a hard limit. It is impossible to add unlimited CPU and memory to a single server.

    Vertical scaling, also known as scaling up, is the scaling by adding more power (CPU, RAM, DISK, etc.) to an existing machine. There are some powerful database servers. According to Amazon Relational Database Service (RDS) [12], you can get a database server with 24 TB of RAM. This kind of powerful database server could store and handle lots of data. For example, stackoverflow.com in 2013 had over 10 million monthly unique visitors, but it only had 1 master database [13]. However, vertical scaling comes with some serious drawbacks:

    • You can add more CPU, RAM, etc. to your database server, but there are hardware limits. If you have a large user base, a single server is not enough.

    • Greater risk of single point of failures.
    • The overall cost of vertical scaling is high. Powerful servers are much more expensive.

  • Vertical scaling does not have failover and redundancy. If one server goes down, the website/app goes down with it completely.

Horizontal scaling is more desirable for large scale applications due to the limitations of vertical scaling.

Database replication

A master database generally only supports write operations. A slave database gets copies of the data from the master database and only supports read operations. All the data-modifying commands like insert, delete, or update must be sent to the master database. Most applications require a much higher ratio of reads to writes; thus, the number of slave databases in a system is usually larger than the number of master databases.

Advantages of database replication:

  • Better performance: In the master-slave model, all writes and updates happen in master nodes; whereas, read operations are distributed across slave nodes. This model improves performance because it allows more queries to be processed in parallel.
  • Reliability: If one of your database servers is destroyed by a natural disaster, such as a typhoon or an earthquake, data is still preserved. You do not need to worry about data loss because data is replicated across multiple locations.
  • High availability: By replicating data across different locations, your website remains in operation even if a database is offline as you can access data stored in another database server.
  • If only one slave database is available and it goes offline, read operations will be directed to the master database temporarily. As soon as the issue is found, a new slave database will replace the old one. In case multiple slave databases are available, read operations are redirected to other healthy slave databases. A new database server will replace the old one.
  • If the master database goes offline, a slave database will be promoted to be the new master. All the database operations will be temporarily executed on the new master database. A new slave database will replace the old one for data replication immediately. In production systems, promoting a new master is more complicated as the data in a slave database might not be up to date. The missing data needs to be updated by running data recovery scripts. Although some other replication methods like multi-masters and circular replication could help, those setups are more complicated; and their discussions are beyond the scope of this book

Cache

A cache is a temporary storage area that stores the result of expensive responses or frequently accessed data in memory so that subsequent requests are served more quickly. Every time a new web page loads, one or more database calls are executed to fetch data. The application performance is greatly affected by calling the database repeatedly. The cache can mitigate this problem.

The cache tier is a temporary data store layer, much faster than the database. The benefits of having a separate cache tier include better system performance, ability to reduce database workloads, and the ability to scale the cache tier independently.

After receiving a request, a web server first checks if the cache has the available response. If it has, it sends data back to the client. If not, it queries the database, stores the response in cache, and sends it back to the client. This caching strategy is called a read-through cache. Other caching strategies are available depending on the data type, size, and access patterns.

Cache considerations

Here are a few considerations for using a cache system:

  • Decide when to use cache. Consider using cache when data is read frequently but modified infrequently. Since cached data is stored in volatile memory, a cache server is not ideal for persisting data. For instance, if a cache server restarts, all the data in memory is lost. Thus, important data should be saved in persistent data stores.
  • Expiration policy. It is a good practice to implement an expiration policy. Once cached data is expired, it is removed from the cache. When there is no expiration policy, cached data will be stored in the memory permanently. It is advisable not to make the expiration date too short as this will cause the system to reload data from the database too frequently. Meanwhile, it is advisable not to make the expiration date too long as the data can become stale.
  • Consistency: This involves keeping the data store and the cache in sync. Inconsistency can happen because data-modifying operations on the data store and cache are not in a single transaction. When scaling across multiple regions, maintaining consistency between the data store and cache is challenging.
  • Mitigating failures: A single cache server represents a potential single point of failure (SPOF), defined in Wikipedia as follows: “A single point of failure (SPOF) is a part of a system that, if it fails, will stop the entire system from working” [8]. As a result, multiple cache servers across different data centers are recommended to avoid SPOF. Another recommended approach is to overprovision the required memory by certain percentages. This provides a buffer as the memory usage increases.
  • Eviction Policy: Once the cache is full, any requests to add items to the cache might cause existing items to be removed. This is called cache eviction. Least-recently-used (LRU) is the most popular cache eviction policy. Other eviction policies, such as the Least Frequently Used (LFU) or First in First Out (FIFO), can be adopted to satisfy different use cases.

CDN(Content Delivery Network)

A CDN is a network of geographically dispersed servers used to deliver static content. CDN servers cache static content like images, videos, CSS, JavaScript files, etc.

Dynamic content caching is a relatively new concept and beyond the scope of this book. It enables the caching of HTML pages that are based on request path, query strings, cookies, and request headers. Refer to the article mentioned in reference material [9] for more about this. This book focuses on how to use CDN to cache static content.

Here is how CDN works at the high-level: when a user visits a website, a CDN server closest to the user will deliver static content. Intuitively, the further users are from CDN servers, the slower the website loads. For example, if CDN servers are in San Francisco, users in Los Angeles will get content faster than users in Europe. Figure 1-9 is a great example that shows how CDN improves load time.

AWS CDN Locations

CDN considerations

  • Cost: CDNs are run by third-party providers, and you are charged for data transfers in and out of the CDN. Caching infrequently used assets provides no significant benefits so you should consider moving them out of the CDN.
  • Setting an appropriate cache expiry: For time-sensitive content, setting a cache expiry time is important. The cache expiry time should neither be too long nor too short. If it is too long, the content might no longer be fresh. If it is too short, it can cause repeat reloading of content from origin servers to the CDN.
  • CDN fallback: You should consider how your website/application copes with CDN failure. If there is a temporary CDN outage, clients should be able to detect the problem and request resources from the origin.
  • Invalidating files: You can remove a file from the CDN before it expires by performing one of the following operations:
    • Invalidate the CDN object using APIs provided by CDN vendors.
    • Use object versioning to serve a different version of the object. To version an object, you can add a parameter to the URL, such as a version number. For example, version number 2 is added to the query string: image.png?v=2.

 

Stateless web tier

Now it is time to consider scaling the web tier horizontally. For this, we need to move state (for instance user session data) out of the web tier. A good practice is to store session data in the persistent storage such as relational database or NoSQL. Each web server in the cluster can access state data from databases. This is called stateless web tier.

In this stateless architecture, HTTP requests from users can be sent to any web servers, which fetch state data from a shared data store. State data is stored in a shared data store and kept out of web servers. A stateless system is simpler, more robust, and scalable.

Here is the design with a stateless web tier.

Here we move the session data out of the web tier and store them in the persistent data store. The shared data store could be a relational database, Memcached/Redis, NoSQL, etc. The NoSQL data store is chosen as it is easy to scale. Autoscaling means adding or removing web servers automatically based on the traffic load. After the state data is removed out of web servers, auto-scaling of the web tier is easily achieved by adding or removing servers based on traffic load.

Your website grows rapidly and attracts a significant number of users internationally. To improve availability and provide a better user experience across wider geographical areas, supporting multiple data centers is crucial.

Stateful architecture

A stateful server and stateless server has some key differences. A stateful server remembers client data (state) from one request to the next. A stateless server keeps no state information.

user A’s session data and profile image are stored in Server 1. To authenticate User A, HTTP requests must be routed to Server 1. If a request is sent to other servers like Server 2, authentication would fail because Server 2 does not contain User A’s session data. Similarly, all HTTP requests from User B must be routed to Server 2; all requests from User C must be sent to Server 3.

The issue is that every request from the same client must be routed to the same server. This can be done with sticky sessions in most load balancers [10]; however, this adds the overhead. Adding or removing servers is much more difficult with this approach. It is also challenging to handle server failures.

 

Data Center

In normal operation, users are geoDNS-routed, also known as geo-routed, to the closest data center, with a split traffic of x% in US-East and (100 – x)% in US-West. geoDNS is a DNS service that allows domain names to be resolved to IP addresses based on the location of a user.

In the event of any significant data center outage, we direct all traffic to a healthy data center.

Several technical challenges must be resolved to achieve multi-data center setup:

  • Traffic redirection: Effective tools are needed to direct traffic to the correct data center. GeoDNS can be used to direct traffic to the nearest data center depending on where a user is located.
  • Data synchronization: Users from different regions could use different local databases or caches. In failover cases, traffic might be routed to a data center where data is unavailable. A common strategy is to replicate data across multiple data centers. A previous study shows how Netflix implements asynchronous multi-data center replication.
  • Test and deployment: With multi-data center setup, it is important to test your website/application at different locations. Automated deployment tools are vital to keep services consistent through all the data centers.

Message Queue

A message queue is a durable component, stored in memory, that supports asynchronous communication. It serves as a buffer and distributes asynchronous requests. The basic architecture of a message queue is simple. Input services, called producers/publishers, create messages, and publish them to a message queue. Other services or servers, called consumers/subscribers, connect to the queue, and perform actions defined by the messages.

Decoupling makes the message queue a preferred architecture for building a scalable and reliable application. With the message queue, the producer can post a message to the queue when the consumer is unavailable to process it. The consumer can read messages from the queue even when the producer is unavailable.

The producer and the consumer can be scaled independently. When the size of the queue becomes large, more workers are added to reduce the processing time. However, if the queue is empty most of the time, the number of workers can be reduced.

Logging, metrics, automation

Logging: Monitoring error logs is important because it helps to identify errors and problems in the system. You can monitor error logs at per server level or use tools to aggregate them to a centralized service for easy search and viewing.

Metrics: Collecting different types of metrics help us to gain business insights and understand the health status of the system. Some of the following metrics are useful:

  • Host level metrics: CPU, Memory, disk I/O, etc.
  • Aggregated level metrics: for example, the performance of the entire database tier, cache tier, etc.
  • Key business metrics: daily active users, retention, revenue, etc.

Automation: When a system gets big and complex, we need to build or leverage automation tools to improve productivity. Continuous integration is a good practice, in which each code check-in is verified through automation, allowing teams to detect problems early. Besides, automating your build, test, deploy process, etc. could improve developer productivity significantly.

 

Millions of users and beyond

Scaling a system is an iterative process. Iterating on what we have learned in this chapter could get us far. More fine-tuning and new strategies are needed to scale beyond millions of users. For example, you might need to optimize your system and decouple the system to even smaller services. All the techniques learned in this chapter should provide a good foundation to tackle new challenges. To conclude this chapter, we provide a summary of how we scale our system to support millions of users:

  • Keep web tier stateless
  • Build redundancy at every tier
  • Cache data as much as you can
  • Support multiple data centers
  • Host static assets in CDN
  • Scale your data tier by sharding
  • Split tiers into individual services
  • Monitor your system and use automation tools
March 5, 2018
Prev
Next