You have a clear problem statement. The stakeholders agree on what needs to be built. Now comes the part where most teams either get it right or waste three months heading in the wrong direction: finding the solution.
This is post #3 in the How It Works series. We are building a task management application, something like a simplified Trello, and using it as a running example. In the previous post, we defined the problem. Now we figure out how to solve it.
Before writing a single line of code, study what already exists. Trello, Jira, Asana, Monday.com, and dozens of other tools have already solved variations of this problem. Your job is not to ignore them. Your job is to learn from them.
Open each one. Use them. Take notes on:
Senior engineers spend more time studying existing systems than junior engineers expect. This research saves you from building features that already exist in better form elsewhere.
Once you understand the landscape, you need to make architectural decisions. For our task management app, here are the real trade-offs a small team faces:
Monolith vs. Microservices: You are a small team building an MVP. Pick a monolith. Microservices add deployment complexity, network latency debugging, and distributed system headaches that a three-person team does not need. You can extract services later when you actually hit scaling problems.
SQL vs. NoSQL: Task data is relational. Tasks belong to boards, boards belong to users, users belong to teams. PostgreSQL gives you ACID transactions, mature tooling, and JOINs that make this straightforward. NoSQL shines for unstructured data at massive scale, which is not where you are starting.
REST vs. GraphQL: REST is simpler to implement, easier to cache, and understood by every developer on the planet. GraphQL is powerful when clients need flexible queries across deeply nested data. For a task app with predictable data shapes, REST wins on simplicity.
The pattern here is clear: choose the simpler option unless you have a specific, measurable reason not to.
Your tech stack should be driven by three factors: what your team already knows, what the project requires, and what has a strong ecosystem.
For our task management app, here is a defensible stack:
Notice that none of these choices are exotic. That is intentional. Every technology listed has years of Stack Overflow answers, production war stories, and hiring pools behind it.
Before committing to a full build, write a spike. A spike is a small, throwaway prototype that tests your riskiest assumption. For our app, the riskiest assumption is real-time task updates: when one user moves a card, every other user on that board should see it instantly.
Start with a simple REST endpoint that serves tasks, then layer WebSocket support on top.
Java Spring Boot spike — a basic task controller:
@RestController
@RequestMapping("/api/tasks")
public class TaskController {
private final List tasks = new ArrayList<>(List.of(
new Task(1L, "Set up CI/CD pipeline", "IN_PROGRESS"),
new Task(2L, "Design database schema", "TODO"),
new Task(3L, "Write API documentation", "DONE")
));
@GetMapping
public ResponseEntity> getTasks() {
return ResponseEntity.ok(tasks);
}
@PostMapping
public ResponseEntity createTask(@RequestBody Task task) {
task.setId((long) tasks.size() + 1);
tasks.add(task);
return ResponseEntity.status(HttpStatus.CREATED).body(task);
}
}
Python Flask spike — the same endpoint:
from flask import Flask, jsonify, request
app = Flask(__name__)
tasks = [
{"id": 1, "title": "Set up CI/CD pipeline", "status": "IN_PROGRESS"},
{"id": 2, "title": "Design database schema", "status": "TODO"},
{"id": 3, "title": "Write API documentation", "status": "DONE"},
]
@app.route("/api/tasks", methods=["GET"])
def get_tasks():
return jsonify(tasks), 200
@app.route("/api/tasks", methods=["POST"])
def create_task():
task = request.get_json()
task["id"] = len(tasks) + 1
tasks.append(task)
return jsonify(task), 201
if __name__ == "__main__":
app.run(debug=True)
Both spikes prove you can serve and create tasks through an API. The next step would be adding a WebSocket endpoint to test real-time updates with your frontend. If the spike works, you have validated your riskiest assumption. If it fails, you found out in two days instead of two months.
Every significant architectural decision should be recorded in an Architecture Decision Record (ADR). ADRs capture what you decided, why, and what the consequences are. Six months from now, when someone asks “why PostgreSQL over MongoDB?”, the ADR answers that question.
Here is the standard format:
# ADR-003: Use PostgreSQL for primary data store ## Status Accepted ## Context Our task management app has relational data (users, teams, boards, tasks). The team has production experience with PostgreSQL. We need ACID transactions for task state changes. ## Decision We will use PostgreSQL 15 as the primary database. ## Consequences - Positive: Strong relational model, mature tooling, team familiarity. - Negative: Vertical scaling limits compared to distributed NoSQL. - Mitigation: Add read replicas if query volume grows beyond single-node capacity.
Keep ADRs in your repository, usually in a docs/adr/ directory. They are version-controlled, searchable, and become part of your project’s institutional memory.
Pick boring technology. The latest JavaScript framework with 200 GitHub stars is not what you want in production. Proven tools like Spring Boot, Django, PostgreSQL, and React have survived years of real-world abuse. They have known failure modes, established best practices, and communities that have already solved the problem you will hit at 2 AM on a Saturday. Boring technology lets you focus on your business problem instead of fighting your tools.
In the next post, we will take our chosen stack and start designing the system architecture: data models, API contracts, and component boundaries.
March 19, 2020This is post #2 in our How It Works series. Our running example is building a task management app — a simplified Trello for small dev teams. In this post, we cover the step that separates experienced engineers from everyone else: understanding the problem before writing a single line of code.
The number one mistake developers make is jumping straight to code. Someone mentions an idea and within an hour there is a half-built prototype with zero clarity on what the software actually needs to do.
Before you open your editor, answer two questions:
If you cannot answer both clearly, you are not ready to build. Every hour spent understanding the problem saves ten hours of rework later.
Requirements come from the people who will use your software. Proven techniques:
Ask “What are you trying to accomplish?” not “What feature do you want?” For our task app, interviews revealed a consistent theme: nobody knew who was working on what.
Distill your findings into a single, clear problem statement — specific, measurable, and free of solution language.
Bad: “We need a Kanban board.”
Good: “Small development teams lose track of who is working on what, leading to duplicated effort and missed deadlines.”
The bad version assumes a solution. The good version describes the pain. Write it down and put it somewhere visible — every decision should trace back to it.
Every project operates within boundaries. Map them early:
Technical: Budget, timeline, team size, tech stack knowledge, infrastructure limits.
Business: Compliance requirements (GDPR, SOC 2), required integrations (Slack, GitHub), expected user base size.
For our app: 2 developers, 6-week timeline, GitHub integration required, targeting teams of 5–20 people.
User stories translate the problem into buildable units: “As a [role], I want [feature], so that [benefit].”
Modeling stories as structured data ensures every story is complete and traceable:
public class UserStory {
private String role;
private String action;
private String benefit;
private List<String> acceptanceCriteria;
public UserStory(String role, String action, String benefit) {
this.role = role;
this.action = action;
this.benefit = benefit;
this.acceptanceCriteria = new ArrayList<>();
}
public void addCriteria(String criteria) {
this.acceptanceCriteria.add(criteria);
}
public String format() {
return String.format(
"As a %s, I want to %s, so that %s.",
role, action, benefit
);
}
public static void main(String[] args) {
UserStory story = new UserStory(
"team lead",
"create tasks and assign them to team members",
"everyone knows who is responsible for what"
);
story.addCriteria("Lead can create a task with title and description");
story.addCriteria("Lead can assign a task to any team member");
story.addCriteria("Assigned member receives a notification");
System.out.println(story.format());
}
}
User stories tell you what to build. Acceptance criteria tell you when it is done — testable conditions that leave no room for ambiguity.
For “As a developer, I want to update my task status,” the criteria might be:
The best way to ensure criteria are testable is to write them as executable tests:
import pytest
from enum import Enum
from datetime import datetime
class TaskStatus(Enum):
TODO = "TODO"
IN_PROGRESS = "IN_PROGRESS"
IN_REVIEW = "IN_REVIEW"
DONE = "DONE"
class Task:
def __init__(self, title, assignee):
self.title = title
self.assignee = assignee
self.status = TaskStatus.TODO
self.history = []
def update_status(self, new_status, changed_by):
if changed_by != self.assignee and changed_by.role != "lead":
raise PermissionError("Only assignee or lead can update")
self.history.append({
"from": self.status, "to": new_status,
"by": changed_by.name, "at": datetime.now()
})
self.status = new_status
class User:
def __init__(self, name, role="developer"):
self.name = name
self.role = role
def test_task_starts_as_todo():
task = Task("Build login page", User("Alice"))
assert task.status == TaskStatus.TODO
def test_assignee_can_update_status():
alice = User("Alice")
task = Task("Build login page", alice)
task.update_status(TaskStatus.IN_PROGRESS, alice)
assert task.status == TaskStatus.IN_PROGRESS
def test_lead_can_update_any_task():
alice = User("Alice")
lead = User("Bob", role="lead")
task = Task("Build login page", alice)
task.update_status(TaskStatus.IN_REVIEW, lead)
assert task.status == TaskStatus.IN_REVIEW
def test_non_assignee_cannot_update():
task = Task("Build login page", User("Alice"))
with pytest.raises(PermissionError):
task.update_status(TaskStatus.DONE, User("Charlie"))
def test_status_change_records_history():
alice = User("Alice")
task = Task("Build login page", alice)
task.update_status(TaskStatus.IN_PROGRESS, alice)
assert len(task.history) == 1
assert task.history[0]["from"] == TaskStatus.TODO
assert task.history[0]["to"] == TaskStatus.IN_PROGRESS
Each test maps directly to an acceptance criterion. When all tests pass, the story is done. No debate, no ambiguity.
The projects that succeed spend more time in this phase than you expect. The projects that fail skip it entirely. Resist the urge to start coding immediately. When you feel the pull to “just start building,” write a problem statement instead. If you cannot write one that is clear and specific, you do not understand the problem yet — and your code will not solve it.
In the next post, we move into Finding the Solution — where we explore possible approaches, evaluate trade-offs, and choose the right path forward.
Most tutorials teach you how to code. They walk you through syntax, frameworks, and algorithms. That’s valuable, but it’s only a fraction of what building real software demands. When you sit down on a professional team and someone says “we need to build this,” the gap between writing code and shipping a product becomes painfully obvious.
Building software is about solving problems systematically. It requires understanding what users actually need, not just what they say they want. It means planning before you write a single line of code, testing before you ship, and iterating after you launch. The best engineers aren’t the ones who write the most clever code. They’re the ones who consistently deliver working software that solves real problems.
This series exists to bridge that gap. We’re going to walk through the entire software development lifecycle, step by step, using a real project as our running example. By the end, you won’t just know how to code a feature. You’ll understand how professional engineering teams take a raw idea and turn it into production software.
Every piece of software you use, from a mobile banking app to a command-line tool, went through some version of the following process. Some teams formalize every step. Others move fast and compress them. But the stages are always there, whether you acknowledge them or not.
Here’s the full lifecycle we’ll cover in this series:
Each of these stages gets its own post in this series. We’ll go deep on each one, with concrete examples and practical techniques you can apply immediately.
Theory is useless without practice. So throughout this series, we’ll build a task management application — think of it as a simplified version of Trello or Jira. Users can create tasks, assign them to team members, track their status, and move them through a workflow.
Why a task management app? Because it’s complex enough to hit every stage of the lifecycle, but simple enough that you won’t get lost in domain-specific details. You already know what task management is. That lets us focus on the process of building software rather than getting bogged down explaining the domain.
At the core of any application is the domain model — the data structures that represent the real-world concepts your software manages. For our task management app, the central entity is a Task. Let’s look at how we’d model it in two languages.
public class Task {
private Long id;
private String title;
private String description;
private String status;
private String assignee;
public Task() {
}
public Task(Long id, String title, String description, String status, String assignee) {
this.id = id;
this.title = title;
this.description = description;
this.status = status;
this.assignee = assignee;
}
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
public String getDescription() {
return description;
}
public void setDescription(String description) {
this.description = description;
}
public String getStatus() {
return status;
}
public void setStatus(String status) {
this.status = status;
}
public String getAssignee() {
return assignee;
}
public void setAssignee(String assignee) {
this.assignee = assignee;
}
@Override
public String toString() {
return "Task{id=" + id + ", title='" + title + "', status='" + status + "', assignee='" + assignee + "'}";
}
}
class Task:
"""Represents a task in our task management application."""
VALID_STATUSES = ["TODO", "IN_PROGRESS", "IN_REVIEW", "DONE"]
def __init__(self, task_id: int, title: str, description: str = "",
status: str = "TODO", assignee: str = None):
self.task_id = task_id
self.title = title
self.description = description
self.status = status
self.assignee = assignee
def assign_to(self, assignee: str):
"""Assign this task to a team member."""
self.assignee = assignee
def update_status(self, new_status: str):
"""Move the task to a new status."""
if new_status not in self.VALID_STATUSES:
raise ValueError(f"Invalid status: {new_status}. Must be one of {self.VALID_STATUSES}")
self.status = new_status
def __repr__(self):
return f"Task(id={self.task_id}, title='{self.title}', status='{self.status}', assignee='{self.assignee}')"
# Example usage
if __name__ == "__main__":
task = Task(task_id=1, title="Design database schema", description="Define tables for tasks and users")
task.assign_to("Alice")
task.update_status("IN_PROGRESS")
print(task)
# Output: Task(id=1, title='Design database schema', status='IN_PROGRESS', assignee='Alice')
Notice that both implementations capture the same core concepts: a task has an identifier, a title, a status that changes over time, and an assignee. The Java version follows the traditional getter/setter pattern, while the Python version is more concise and includes basic validation. These are not production-ready classes — they are starting points. As we move through the series, we will evolve them into a full application with persistence, APIs, and a user interface.
By the end of this series, you will understand:
This isn’t about any single language or framework. The principles apply whether you’re building with Java and Spring Boot, Python and Django, or anything else. The tools change. The process doesn’t.
This series is for junior to mid-level developers who can write code but feel uncertain about the bigger picture. If you’ve completed tutorials and built personal projects but wonder how professional teams actually operate, this is for you.
You should be comfortable reading Java and Python code. You don’t need to be an expert in either. The code examples are straightforward and well-commented.
If you’re a senior developer, you might still find value here as a refresher or as a resource to share with more junior colleagues. Sometimes it helps to see the process laid out explicitly, especially when you’ve been doing it intuitively for years.
Let’s get started. In the next post, we’ll tackle the first and most overlooked stage of software development: understanding the problem.
March 19, 2020
Encryption is the process of converting readable data (plaintext) into an unreadable format (ciphertext) using a mathematical algorithm and a secret key. Only someone who has the correct key can reverse the process (decryption) and recover the original data.
Real-world analogy: Think of encryption like a lockbox. You put a letter (plaintext) inside a lockbox, close it with a key (encryption), and hand it to someone. Only the person with the matching key can open the box and read the letter (decryption). Anyone who intercepts the locked box sees nothing useful.
Encryption is critical in software development for several reasons:
There are two fundamental types of encryption:
| Type | Keys Used | Speed | Use Case | Example Algorithms |
|---|---|---|---|---|
| Symmetric Encryption | Same key for encrypt and decrypt | Fast | Encrypting large amounts of data | AES, DES, ChaCha20 |
| Asymmetric Encryption | Public key encrypts, private key decrypts | Slow | Key exchange, digital signatures | RSA, ECDSA, Ed25519 |
In practice, most systems use both together. For example, HTTPS uses asymmetric encryption to exchange a symmetric key, then uses that symmetric key to encrypt the actual data. This is called hybrid encryption.
Here is a simple visualization of the encryption process:
Throughout this tutorial, we will explore how Java provides all the tools you need to implement encryption, hashing, and message authentication in your applications.
Java provides a comprehensive set of cryptographic APIs through the Java Cryptography Architecture (JCA) and the Java Cryptography Extension (JCE). These are part of the standard JDK, so you do not need any external libraries for core cryptographic operations.
Here are the key classes and packages you will use:
| Class / Package | Purpose | Example Use |
|---|---|---|
javax.crypto.Cipher |
Core encryption/decryption engine | AES encryption, RSA encryption |
javax.crypto.KeyGenerator |
Generate symmetric keys | Generate AES-256 key |
javax.crypto.SecretKey |
Represents a symmetric key | AES key storage |
javax.crypto.Mac |
Message Authentication Code | HMAC-SHA256 |
java.security.KeyPairGenerator |
Generate public/private key pairs | RSA key pair |
java.security.MessageDigest |
Cryptographic hashing | SHA-256 hash |
java.security.SecureRandom |
Cryptographically secure random numbers | Generating IVs, salts, tokens |
java.security.Signature |
Digital signatures | Sign and verify data |
java.util.Base64 |
Encode/decode binary data as text | Transmitting encrypted data as strings |
The JCA uses a provider model. A provider is a package that implements cryptographic algorithms. The default provider is SunJCE, but you can plug in third-party providers like Bouncy Castle for additional algorithms. In most cases, the built-in providers are sufficient.
The general pattern for using JCA classes follows a consistent factory pattern:
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import java.security.MessageDigest;
import java.security.SecureRandom;
public class JCAOverview {
public static void main(String[] args) throws Exception {
// 1. Cipher -- the core encryption engine
// Pattern: Cipher.getInstance("Algorithm/Mode/Padding")
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
System.out.println("Cipher algorithm: " + cipher.getAlgorithm());
// Output: Cipher algorithm: AES/GCM/NoPadding
// 2. KeyGenerator -- generates symmetric keys
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256); // 256-bit AES key
System.out.println("KeyGenerator algorithm: " + keyGen.getAlgorithm());
// Output: KeyGenerator algorithm: AES
// 3. MessageDigest -- hashing
MessageDigest digest = MessageDigest.getInstance("SHA-256");
System.out.println("MessageDigest algorithm: " + digest.getAlgorithm());
// Output: MessageDigest algorithm: SHA-256
// 4. SecureRandom -- cryptographically strong random numbers
SecureRandom random = new SecureRandom();
byte[] randomBytes = new byte[16];
random.nextBytes(randomBytes);
System.out.println("Generated " + randomBytes.length + " random bytes");
// Output: Generated 16 random bytes
// 5. List available providers
for (java.security.Provider provider : java.security.Security.getProviders()) {
System.out.println("Provider: " + provider.getName() + " v" + provider.getVersion());
}
// Output: Provider: SUN v22.0, Provider: SunJCE v22.0, ...
}
}
Hashing is a one-way process that converts data of any size into a fixed-length string of characters. Unlike encryption, hashing is not reversible — you cannot recover the original data from a hash. The same input always produces the same hash (deterministic), but even a tiny change in input produces a completely different hash (avalanche effect).
Important: Hashing is NOT encryption. Encryption is reversible (with the key); hashing is not. They serve different purposes.
| Algorithm | Output Size | Secure? | Use Case |
|---|---|---|---|
| MD5 | 128 bits (32 hex chars) | No — broken, collisions found | Checksums only (never for security) |
| SHA-1 | 160 bits (40 hex chars) | No — deprecated | Legacy systems only |
| SHA-256 | 256 bits (64 hex chars) | Yes | Data integrity, file verification |
| SHA-512 | 512 bits (128 hex chars) | Yes | High security requirements |
Java provides hashing through the MessageDigest class:
import java.security.MessageDigest;
import java.nio.charset.StandardCharsets;
public class HashingExamples {
public static void main(String[] args) throws Exception {
String data = "Hello, World!";
// SHA-256 hash
String sha256Hash = hash(data, "SHA-256");
System.out.println("SHA-256: " + sha256Hash);
// Output: SHA-256: dffd6021bb2bd5b0af676290809ec3a53191dd81c7f70a4b28688a362182986f
// SHA-512 hash
String sha512Hash = hash(data, "SHA-512");
System.out.println("SHA-512: " + sha512Hash);
// Output: SHA-512: 374d794a95cdcfd8b35993185fef9ba368f160d8daf432d08ba9f1ed1e5abe6cc69291e0fa2fe0006a52570ef18c19def4e617c33ce52ef0a6e5fbe318cb0387
// MD5 (DO NOT use for security purposes)
String md5Hash = hash(data, "MD5");
System.out.println("MD5: " + md5Hash);
// Output: MD5: 65a8e27d8879283831b664bd8b7f0ad4
// Demonstrate avalanche effect -- tiny change, completely different hash
String data2 = "Hello, World?"; // changed ! to ?
String hash2 = hash(data2, "SHA-256");
System.out.println("\nOriginal: " + sha256Hash);
System.out.println("Modified: " + hash2);
// Completely different hashes despite one character difference
}
public static String hash(String data, String algorithm) throws Exception {
MessageDigest digest = MessageDigest.getInstance(algorithm);
byte[] hashBytes = digest.digest(data.getBytes(StandardCharsets.UTF_8));
return bytesToHex(hashBytes);
}
private static String bytesToHex(byte[] bytes) {
StringBuilder hexString = new StringBuilder();
for (byte b : bytes) {
String hex = Integer.toHexString(0xff & b);
if (hex.length() == 1) {
hexString.append('0');
}
hexString.append(hex);
}
return hexString.toString();
}
}
Hashing with Salt: When hashing passwords or sensitive data, you should always add a salt — a random value prepended to the input before hashing. This prevents attackers from using precomputed hash tables (rainbow tables) to crack your hashes.
import java.security.MessageDigest;
import java.security.SecureRandom;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
public class SaltedHashing {
public static void main(String[] args) throws Exception {
String password = "mySecretPassword";
// Generate a random salt (16 bytes = 128 bits)
byte[] salt = generateSalt();
System.out.println("Salt: " + Base64.getEncoder().encodeToString(salt));
// Hash with salt
String hashedPassword = hashWithSalt(password, salt);
System.out.println("Hashed: " + hashedPassword);
// Verify -- same password + same salt = same hash
String verifyHash = hashWithSalt(password, salt);
System.out.println("Verified: " + hashedPassword.equals(verifyHash));
// Output: Verified: true
// Different password = different hash
String wrongHash = hashWithSalt("wrongPassword", salt);
System.out.println("Wrong password match: " + hashedPassword.equals(wrongHash));
// Output: Wrong password match: false
}
public static byte[] generateSalt() {
byte[] salt = new byte[16];
new SecureRandom().nextBytes(salt);
return salt;
}
public static String hashWithSalt(String password, byte[] salt) throws Exception {
MessageDigest digest = MessageDigest.getInstance("SHA-256");
digest.update(salt); // add salt first
byte[] hashBytes = digest.digest(password.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(hashBytes);
}
}
Warning: Even salted SHA-256 is NOT sufficient for password hashing in production. SHA-256 is too fast — attackers can try billions of guesses per second. For passwords, use a dedicated password hashing algorithm like bcrypt, scrypt, or Argon2 (covered in Section 8).
Symmetric encryption uses the same key for both encryption and decryption. It is fast and efficient, making it the standard choice for encrypting data at rest and in transit.
AES (Advanced Encryption Standard) is the most widely used symmetric encryption algorithm. It was adopted by the U.S. government in 2001 and is considered secure with proper key sizes (128, 192, or 256 bits).
When using AES, you must choose a cipher mode and padding scheme. The mode determines how blocks of data are processed:
| Mode | Full Name | IV Required? | Authenticated? | Recommendation |
|---|---|---|---|---|
| ECB | Electronic Codebook | No | No | NEVER use — identical blocks produce identical ciphertext, leaks patterns |
| CBC | Cipher Block Chaining | Yes | No | Acceptable but requires separate HMAC for authentication |
| CTR | Counter | Yes | No | Good for streaming, but no built-in authentication |
| GCM | Galois/Counter Mode | Yes | Yes | Recommended — provides both encryption AND authentication |
Key Concepts:
NoPadding).Let us start with AES key generation:
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;
import java.util.Base64;
public class AESKeyGeneration {
public static void main(String[] args) throws Exception {
// Generate a new AES-256 key
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256); // 128, 192, or 256 bits
SecretKey secretKey = keyGen.generateKey();
// Convert key to Base64 string (for storage)
String encodedKey = Base64.getEncoder().encodeToString(secretKey.getEncoded());
System.out.println("AES Key (Base64): " + encodedKey);
System.out.println("Key length: " + secretKey.getEncoded().length + " bytes");
// Output: Key length: 32 bytes (256 bits)
// Reconstruct key from Base64 string
byte[] decodedKey = Base64.getDecoder().decode(encodedKey);
SecretKey reconstructedKey = new SecretKeySpec(decodedKey, "AES");
System.out.println("Keys match: " + secretKey.equals(reconstructedKey));
// Output: Keys match: true
}
}
Now let us implement the recommended approach: AES-256-GCM. This mode provides both encryption and authentication (it detects if the ciphertext has been tampered with).
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;
import java.security.SecureRandom;
import java.util.Base64;
public class AESGCMEncryption {
private static final int GCM_IV_LENGTH = 12; // 96 bits (recommended for GCM)
private static final int GCM_TAG_LENGTH = 128; // 128 bits authentication tag
public static void main(String[] args) throws Exception {
// Generate AES-256 key
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey secretKey = keyGen.generateKey();
String plaintext = "This is a secret message that needs to be encrypted.";
System.out.println("Original: " + plaintext);
// Encrypt
String encrypted = encrypt(plaintext, secretKey);
System.out.println("Encrypted: " + encrypted);
// Decrypt
String decrypted = decrypt(encrypted, secretKey);
System.out.println("Decrypted: " + decrypted);
// Verify
System.out.println("Match: " + plaintext.equals(decrypted));
// Output: Match: true
// Encrypting the same text again produces DIFFERENT ciphertext (because of random IV)
String encrypted2 = encrypt(plaintext, secretKey);
System.out.println("\nSame plaintext, different ciphertext:");
System.out.println("Encryption 1: " + encrypted);
System.out.println("Encryption 2: " + encrypted2);
System.out.println("Same ciphertext? " + encrypted.equals(encrypted2));
// Output: Same ciphertext? false (because IV is different each time)
}
public static String encrypt(String plaintext, SecretKey key) throws Exception {
// Generate a random IV
byte[] iv = new byte[GCM_IV_LENGTH];
new SecureRandom().nextBytes(iv);
// Initialize cipher for encryption
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec gcmSpec = new GCMParameterSpec(GCM_TAG_LENGTH, iv);
cipher.init(Cipher.ENCRYPT_MODE, key, gcmSpec);
// Encrypt
byte[] ciphertext = cipher.doFinal(plaintext.getBytes("UTF-8"));
// Prepend IV to ciphertext (IV is needed for decryption but is not secret)
byte[] ivAndCiphertext = new byte[iv.length + ciphertext.length];
System.arraycopy(iv, 0, ivAndCiphertext, 0, iv.length);
System.arraycopy(ciphertext, 0, ivAndCiphertext, iv.length, ciphertext.length);
return Base64.getEncoder().encodeToString(ivAndCiphertext);
}
public static String decrypt(String encryptedBase64, SecretKey key) throws Exception {
byte[] ivAndCiphertext = Base64.getDecoder().decode(encryptedBase64);
// Extract IV from the beginning
byte[] iv = new byte[GCM_IV_LENGTH];
System.arraycopy(ivAndCiphertext, 0, iv, 0, iv.length);
// Extract ciphertext (rest of the bytes)
byte[] ciphertext = new byte[ivAndCiphertext.length - iv.length];
System.arraycopy(ivAndCiphertext, iv.length, ciphertext, 0, ciphertext.length);
// Initialize cipher for decryption
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec gcmSpec = new GCMParameterSpec(GCM_TAG_LENGTH, iv);
cipher.init(Cipher.DECRYPT_MODE, key, gcmSpec);
// Decrypt
byte[] plaintext = cipher.doFinal(ciphertext);
return new String(plaintext, "UTF-8");
}
}
For reference, here is AES-CBC encryption (still commonly used in legacy systems). Note that CBC mode requires a separate HMAC step for authentication, which is why GCM is preferred:
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.IvParameterSpec;
import java.security.SecureRandom;
import java.util.Base64;
public class AESCBCEncryption {
private static final int IV_SIZE = 16; // 128 bits for AES block size
public static String encrypt(String plaintext, SecretKey key) throws Exception {
// Generate random IV
byte[] iv = new byte[IV_SIZE];
new SecureRandom().nextBytes(iv);
// Initialize cipher
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.ENCRYPT_MODE, key, new IvParameterSpec(iv));
// Encrypt
byte[] ciphertext = cipher.doFinal(plaintext.getBytes("UTF-8"));
// Prepend IV to ciphertext
byte[] result = new byte[iv.length + ciphertext.length];
System.arraycopy(iv, 0, result, 0, iv.length);
System.arraycopy(ciphertext, 0, result, iv.length, ciphertext.length);
return Base64.getEncoder().encodeToString(result);
}
public static String decrypt(String encryptedBase64, SecretKey key) throws Exception {
byte[] data = Base64.getDecoder().decode(encryptedBase64);
// Extract IV
byte[] iv = new byte[IV_SIZE];
System.arraycopy(data, 0, iv, 0, iv.length);
// Extract ciphertext
byte[] ciphertext = new byte[data.length - iv.length];
System.arraycopy(data, iv.length, ciphertext, 0, ciphertext.length);
// Decrypt
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
cipher.init(Cipher.DECRYPT_MODE, key, new IvParameterSpec(iv));
return new String(cipher.doFinal(ciphertext), "UTF-8");
}
public static void main(String[] args) throws Exception {
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey key = keyGen.generateKey();
String message = "AES-CBC encryption example";
String encrypted = encrypt(message, key);
String decrypted = decrypt(encrypted, key);
System.out.println("Original: " + message);
System.out.println("Encrypted: " + encrypted);
System.out.println("Decrypted: " + decrypted);
// Output:
// Original: AES-CBC encryption example
// Encrypted: (Base64 string - different each time due to random IV)
// Decrypted: AES-CBC encryption example
}
}
Asymmetric encryption uses a pair of keys: a public key and a private key. Data encrypted with the public key can only be decrypted with the corresponding private key, and vice versa.
Real-world analogy: Think of a mailbox. Anyone can drop a letter into the mailbox (encrypt with the public key), but only the person with the mailbox key can open it and read the letters (decrypt with the private key).
RSA (Rivest-Shamir-Adleman) is the most widely used asymmetric algorithm. It is significantly slower than AES, so it is typically used for:
| Feature | Symmetric (AES) | Asymmetric (RSA) |
|---|---|---|
| Keys | 1 shared key | 2 keys (public + private) |
| Speed | Very fast | ~100x slower |
| Key distribution | Hard (must share secretly) | Easy (public key is public) |
| Data size limit | No limit | Limited by key size |
| Use case | Bulk data encryption | Key exchange, signatures |
import javax.crypto.Cipher;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.PrivateKey;
import java.security.PublicKey;
import java.util.Base64;
public class RSAEncryption {
public static void main(String[] args) throws Exception {
// Generate RSA key pair (2048 bits minimum, 4096 for high security)
KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA");
keyGen.initialize(2048);
KeyPair keyPair = keyGen.generateKeyPair();
PublicKey publicKey = keyPair.getPublic();
PrivateKey privateKey = keyPair.getPrivate();
System.out.println("Public key format: " + publicKey.getFormat());
// Output: Public key format: X.509
System.out.println("Private key format: " + privateKey.getFormat());
// Output: Private key format: PKCS#8
// Encrypt with public key
String plaintext = "Secret message for RSA";
String encrypted = encrypt(plaintext, publicKey);
System.out.println("Encrypted: " + encrypted);
// Decrypt with private key
String decrypted = decrypt(encrypted, privateKey);
System.out.println("Decrypted: " + decrypted);
// Output: Decrypted: Secret message for RSA
}
public static String encrypt(String plaintext, PublicKey publicKey) throws Exception {
// Use RSA with OAEP padding (recommended over PKCS1)
Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
cipher.init(Cipher.ENCRYPT_MODE, publicKey);
byte[] ciphertext = cipher.doFinal(plaintext.getBytes("UTF-8"));
return Base64.getEncoder().encodeToString(ciphertext);
}
public static String decrypt(String encryptedBase64, PrivateKey privateKey) throws Exception {
Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA-256AndMGF1Padding");
cipher.init(Cipher.DECRYPT_MODE, privateKey);
byte[] plaintext = cipher.doFinal(Base64.getDecoder().decode(encryptedBase64));
return new String(plaintext, "UTF-8");
}
}
Digital signatures work in the opposite direction from encryption. You sign with your private key (proving you are the author) and anyone can verify with your public key (confirming the data was not modified).
import java.security.*;
import java.util.Base64;
public class DigitalSignatureExample {
public static void main(String[] args) throws Exception {
// Generate key pair
KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA");
keyGen.initialize(2048);
KeyPair keyPair = keyGen.generateKeyPair();
String message = "This document was signed by me.";
// Sign with private key
String signature = sign(message, keyPair.getPrivate());
System.out.println("Signature: " + signature);
// Verify with public key
boolean isValid = verify(message, signature, keyPair.getPublic());
System.out.println("Signature valid: " + isValid);
// Output: Signature valid: true
// Tamper with the message
boolean isTamperedValid = verify("This document was NOT signed by me.", signature, keyPair.getPublic());
System.out.println("Tampered message valid: " + isTamperedValid);
// Output: Tampered message valid: false
}
public static String sign(String data, PrivateKey privateKey) throws Exception {
Signature signer = Signature.getInstance("SHA256withRSA");
signer.initSign(privateKey);
signer.update(data.getBytes("UTF-8"));
byte[] signatureBytes = signer.sign();
return Base64.getEncoder().encodeToString(signatureBytes);
}
public static boolean verify(String data, String signatureBase64, PublicKey publicKey) throws Exception {
Signature verifier = Signature.getInstance("SHA256withRSA");
verifier.initVerify(publicKey);
verifier.update(data.getBytes("UTF-8"));
byte[] signatureBytes = Base64.getDecoder().decode(signatureBase64);
return verifier.verify(signatureBytes);
}
}
Important: Base64 is NOT encryption. It is an encoding scheme that converts binary data into a text representation using 64 printable ASCII characters. Anyone can decode Base64 — there is no key, no secret, and no security. It is completely reversible by anyone.
Base64 is used when you need to transmit binary data (like encrypted bytes, images, or files) over channels that only support text (like JSON, XML, email, or URLs).
Java 8+ provides the java.util.Base64 class with three variants:
| Encoder/Decoder | Character Set | Use Case |
|---|---|---|
Base64.getEncoder() |
A-Z, a-z, 0-9, +, / | General purpose (may include line breaks with MIME) |
Base64.getUrlEncoder() |
A-Z, a-z, 0-9, -, _ | URL-safe (no +, /, or =) |
Base64.getMimeEncoder() |
Same as basic + line breaks | Email attachments (MIME format) |
import java.util.Base64;
import java.nio.charset.StandardCharsets;
public class Base64Examples {
public static void main(String[] args) {
String original = "Hello, Java Encryption!";
// Basic encoding/decoding
String encoded = Base64.getEncoder().encodeToString(original.getBytes(StandardCharsets.UTF_8));
System.out.println("Encoded: " + encoded);
// Output: Encoded: SGVsbG8sIEphdmEgRW5jcnlwdGlvbiE=
String decoded = new String(Base64.getDecoder().decode(encoded), StandardCharsets.UTF_8);
System.out.println("Decoded: " + decoded);
// Output: Decoded: Hello, Java Encryption!
// URL-safe encoding (replaces + with - and / with _)
byte[] binaryData = {(byte) 0xFF, (byte) 0xFE, (byte) 0xFD, (byte) 0xFC};
String urlSafe = Base64.getUrlEncoder().encodeToString(binaryData);
System.out.println("URL-safe: " + urlSafe);
// Output: URL-safe: __79_A==
// URL-safe without padding (no trailing = signs)
String urlSafeNoPadding = Base64.getUrlEncoder().withoutPadding().encodeToString(binaryData);
System.out.println("URL-safe (no padding): " + urlSafeNoPadding);
// Output: URL-safe (no padding): __79_A
// Encoding binary data (like encrypted bytes) for storage as text
byte[] encryptedBytes = new byte[]{0x12, 0x34, 0x56, 0x78, (byte) 0x9A, (byte) 0xBC};
String storable = Base64.getEncoder().encodeToString(encryptedBytes);
System.out.println("Binary as text: " + storable);
// Output: Binary as text: EjRWeJq8
// Decode back to original bytes
byte[] restored = Base64.getDecoder().decode(storable);
System.out.println("Bytes match: " + java.util.Arrays.equals(encryptedBytes, restored));
// Output: Bytes match: true
}
}
An HMAC is a specific type of message authentication code that combines a cryptographic hash function with a secret key. It provides two guarantees:
Unlike a plain hash, an HMAC requires a secret key. This means an attacker cannot forge a valid HMAC without knowing the key. This is why HMACs are used in API authentication (like AWS request signing), JWT tokens, webhook verification, and session tokens.
Common HMAC algorithms:
| Algorithm | Output Size | Use Case |
|---|---|---|
| HmacSHA256 | 256 bits | Most common — API signing, JWTs, general purpose |
| HmacSHA384 | 384 bits | Higher security requirement |
| HmacSHA512 | 512 bits | Maximum security, slightly slower |
| HmacMD5 | 128 bits | Do NOT use — MD5 is broken |
import javax.crypto.Mac;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
public class HMACExample {
public static void main(String[] args) throws Exception {
// Generate a key for HMAC
KeyGenerator keyGen = KeyGenerator.getInstance("HmacSHA256");
SecretKey hmacKey = keyGen.generateKey();
String message = "Order #12345: $99.99 charged to card ending in 4242";
// Generate HMAC
String hmac = generateHMAC(message, hmacKey);
System.out.println("Message: " + message);
System.out.println("HMAC: " + hmac);
// Verify HMAC -- same message + same key = same HMAC
String verifyHmac = generateHMAC(message, hmacKey);
System.out.println("Valid: " + hmac.equals(verifyHmac));
// Output: Valid: true
// Tampered message produces different HMAC
String tamperedMessage = "Order #12345: $9999.99 charged to card ending in 4242";
String tamperedHmac = generateHMAC(tamperedMessage, hmacKey);
System.out.println("Tampered valid: " + hmac.equals(tamperedHmac));
// Output: Tampered valid: false
// Using a string-based key (common for API integrations)
String apiSecret = "my-api-secret-key-do-not-share";
SecretKeySpec stringKey = new SecretKeySpec(
apiSecret.getBytes(StandardCharsets.UTF_8), "HmacSHA256"
);
String apiHmac = generateHMAC("webhook-payload-data", stringKey);
System.out.println("API HMAC: " + apiHmac);
}
public static String generateHMAC(String data, SecretKey key) throws Exception {
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(key);
byte[] hmacBytes = mac.doFinal(data.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(hmacBytes);
}
}
Practical use case: Webhook verification. When services like Stripe or GitHub send webhooks, they include an HMAC signature in the header. You recompute the HMAC using your secret key and compare it to the received signature. If they match, the webhook is legitimate.
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
public class WebhookVerification {
/**
* Verify a webhook signature.
* The sender computes HMAC-SHA256 of the body using a shared secret
* and sends it in the X-Signature header.
*/
public static boolean verifyWebhook(String payload, String receivedSignature, String secret)
throws Exception {
SecretKeySpec key = new SecretKeySpec(
secret.getBytes(StandardCharsets.UTF_8), "HmacSHA256"
);
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(key);
byte[] computedBytes = mac.doFinal(payload.getBytes(StandardCharsets.UTF_8));
// Convert to hex string for comparison
StringBuilder hex = new StringBuilder();
for (byte b : computedBytes) {
hex.append(String.format("%02x", b));
}
String computedSignature = hex.toString();
// Use constant-time comparison to prevent timing attacks
return java.security.MessageDigest.isEqual(
computedSignature.getBytes(),
receivedSignature.getBytes()
);
}
public static void main(String[] args) throws Exception {
String webhookBody = "{\"event\":\"payment.completed\",\"amount\":9999}";
String sharedSecret = "whsec_test_secret_key";
// Simulate: sender computes signature
SecretKeySpec key = new SecretKeySpec(
sharedSecret.getBytes(StandardCharsets.UTF_8), "HmacSHA256"
);
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(key);
byte[] sigBytes = mac.doFinal(webhookBody.getBytes(StandardCharsets.UTF_8));
StringBuilder sigHex = new StringBuilder();
for (byte b : sigBytes) {
sigHex.append(String.format("%02x", b));
}
// Receiver verifies
boolean valid = verifyWebhook(webhookBody, sigHex.toString(), sharedSecret);
System.out.println("Webhook signature valid: " + valid);
// Output: Webhook signature valid: true
// Tampered payload fails
boolean tampered = verifyWebhook(
"{\"event\":\"payment.completed\",\"amount\":0}",
sigHex.toString(), sharedSecret
);
System.out.println("Tampered webhook valid: " + tampered);
// Output: Tampered webhook valid: false
}
}
Password hashing is a special case that requires algorithms specifically designed to be slow. This might sound counterintuitive, but here is why:
| Approach | Hashes/Second (GPU) | Suitable for Passwords? |
|---|---|---|
| MD5 | ~100 billion | Absolutely not |
| SHA-256 | ~10 billion | No — too fast |
| SHA-256 + salt | ~10 billion | No — still too fast |
| bcrypt | ~10,000 (tunable) | Yes — recommended |
| scrypt | ~1,000 (tunable) | Yes — memory-hard |
| Argon2 | ~100 (tunable) | Yes — winner of PHC, best choice |
bcrypt is the most widely used in Java applications. It automatically handles salt generation and embeds the salt in the output, so you do not need to manage salts separately.
Here is a bcrypt implementation using the jBCrypt library (or Spring Security’s BCryptPasswordEncoder):
// Using Spring Security's BCryptPasswordEncoder (most common in Java projects)
// Add dependency: org.springframework.security:spring-security-crypto
import org.springframework.security.crypto.bcrypt.BCryptPasswordEncoder;
public class BCryptPasswordExample {
public static void main(String[] args) {
BCryptPasswordEncoder encoder = new BCryptPasswordEncoder(12); // work factor = 12
String rawPassword = "MySecureP@ssw0rd!";
// Hash the password (salt is generated automatically)
String hashedPassword = encoder.encode(rawPassword);
System.out.println("Hashed: " + hashedPassword);
// Output: Hashed: $2a$12$LJ/3Xq... (60 characters, different each time)
// Hashing the same password again gives a DIFFERENT hash (different salt)
String hashedPassword2 = encoder.encode(rawPassword);
System.out.println("Hashed again: " + hashedPassword2);
System.out.println("Same hash? " + hashedPassword.equals(hashedPassword2));
// Output: Same hash? false (because salt is different)
// Verify password -- this is how you check login credentials
boolean matches = encoder.matches(rawPassword, hashedPassword);
System.out.println("Correct password: " + matches);
// Output: Correct password: true
boolean wrongMatch = encoder.matches("WrongPassword", hashedPassword);
System.out.println("Wrong password: " + wrongMatch);
// Output: Wrong password: false
}
}
If you are not using Spring Security, here is a pure Java approach using PBKDF2 (Password-Based Key Derivation Function 2), which is built into the JDK:
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.PBEKeySpec;
import java.security.SecureRandom;
import java.util.Base64;
public class PBKDF2PasswordHashing {
private static final int ITERATIONS = 210_000; // OWASP recommendation (2023+)
private static final int KEY_LENGTH = 256; // bits
private static final int SALT_LENGTH = 16; // bytes
public static void main(String[] args) throws Exception {
String password = "MySecureP@ssw0rd!";
// Hash the password
String stored = hashPassword(password);
System.out.println("Stored: " + stored);
// Output: Stored: 210000:BASE64_SALT:BASE64_HASH
// Verify
boolean valid = verifyPassword(password, stored);
System.out.println("Valid: " + valid);
// Output: Valid: true
boolean invalid = verifyPassword("WrongPassword", stored);
System.out.println("Wrong password: " + invalid);
// Output: Wrong password: false
}
public static String hashPassword(String password) throws Exception {
// Generate random salt
byte[] salt = new byte[SALT_LENGTH];
new SecureRandom().nextBytes(salt);
// Hash with PBKDF2
PBEKeySpec spec = new PBEKeySpec(
password.toCharArray(), salt, ITERATIONS, KEY_LENGTH
);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
byte[] hash = factory.generateSecret(spec).getEncoded();
// Store iterations:salt:hash (all needed to verify later)
return ITERATIONS + ":"
+ Base64.getEncoder().encodeToString(salt) + ":"
+ Base64.getEncoder().encodeToString(hash);
}
public static boolean verifyPassword(String password, String stored) throws Exception {
String[] parts = stored.split(":");
int iterations = Integer.parseInt(parts[0]);
byte[] salt = Base64.getDecoder().decode(parts[1]);
byte[] expectedHash = Base64.getDecoder().decode(parts[2]);
// Hash the input password with the same salt and iterations
PBEKeySpec spec = new PBEKeySpec(
password.toCharArray(), salt, iterations, expectedHash.length * 8
);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
byte[] actualHash = factory.generateSecret(spec).getEncoded();
// Constant-time comparison to prevent timing attacks
return java.security.MessageDigest.isEqual(expectedHash, actualHash);
}
}
Encrypting files follows the same AES pattern but processes data in streams to handle large files without loading everything into memory:
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
import javax.crypto.CipherOutputStream;
import javax.crypto.KeyGenerator;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;
import java.io.*;
import java.security.SecureRandom;
public class FileEncryption {
private static final int GCM_IV_LENGTH = 12;
private static final int GCM_TAG_LENGTH = 128;
public static void encryptFile(File inputFile, File outputFile, SecretKey key)
throws Exception {
byte[] iv = new byte[GCM_IV_LENGTH];
new SecureRandom().nextBytes(iv);
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(GCM_TAG_LENGTH, iv));
try (FileOutputStream fos = new FileOutputStream(outputFile)) {
// Write IV first (needed for decryption)
fos.write(iv);
try (FileInputStream fis = new FileInputStream(inputFile);
CipherOutputStream cos = new CipherOutputStream(fos, cipher)) {
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = fis.read(buffer)) != -1) {
cos.write(buffer, 0, bytesRead);
}
}
}
}
public static void decryptFile(File inputFile, File outputFile, SecretKey key)
throws Exception {
try (FileInputStream fis = new FileInputStream(inputFile)) {
// Read IV
byte[] iv = new byte[GCM_IV_LENGTH];
fis.read(iv);
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, key, new GCMParameterSpec(GCM_TAG_LENGTH, iv));
try (CipherInputStream cis = new CipherInputStream(fis, cipher);
FileOutputStream fos = new FileOutputStream(outputFile)) {
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = cis.read(buffer)) != -1) {
fos.write(buffer, 0, bytesRead);
}
}
}
}
public static void main(String[] args) throws Exception {
// Generate key
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
SecretKey key = keyGen.generateKey();
// Create a test file
File original = new File("secret_document.txt");
try (FileWriter fw = new FileWriter(original)) {
fw.write("This is my secret document with sensitive information.");
}
// Encrypt
File encrypted = new File("secret_document.enc");
encryptFile(original, encrypted, key);
System.out.println("File encrypted: " + encrypted.getName());
System.out.println("Original size: " + original.length() + " bytes");
System.out.println("Encrypted size: " + encrypted.length() + " bytes");
// Decrypt
File decrypted = new File("secret_document_decrypted.txt");
decryptFile(encrypted, decrypted, key);
System.out.println("File decrypted: " + decrypted.getName());
// Verify
try (BufferedReader br = new BufferedReader(new FileReader(decrypted))) {
System.out.println("Content: " + br.readLine());
}
// Output: Content: This is my secret document with sensitive information.
// Cleanup
original.delete();
encrypted.delete();
decrypted.delete();
}
}
Generating cryptographically secure tokens for session IDs, API keys, password reset links, and verification codes:
import java.security.SecureRandom;
import java.util.Base64;
public class SecureTokenGenerator {
private static final SecureRandom secureRandom = new SecureRandom();
/**
* Generate a URL-safe token (for password reset links, API keys, etc.)
*/
public static String generateToken(int byteLength) {
byte[] tokenBytes = new byte[byteLength];
secureRandom.nextBytes(tokenBytes);
return Base64.getUrlEncoder().withoutPadding().encodeToString(tokenBytes);
}
/**
* Generate a hex token (for session IDs, CSRF tokens)
*/
public static String generateHexToken(int byteLength) {
byte[] tokenBytes = new byte[byteLength];
secureRandom.nextBytes(tokenBytes);
StringBuilder hex = new StringBuilder();
for (byte b : tokenBytes) {
hex.append(String.format("%02x", b));
}
return hex.toString();
}
/**
* Generate a numeric OTP (for SMS verification, 2FA)
*/
public static String generateOTP(int digits) {
int max = (int) Math.pow(10, digits);
int otp = secureRandom.nextInt(max);
return String.format("%0" + digits + "d", otp);
}
public static void main(String[] args) {
// API key (32 bytes = 256 bits of entropy)
System.out.println("API Key: " + generateToken(32));
// Output: API Key: xK9mPq... (43 characters, URL-safe)
// Session ID (16 bytes = 128 bits)
System.out.println("Session ID: " + generateHexToken(16));
// Output: Session ID: a1b2c3d4e5f6... (32 hex characters)
// Password reset token (24 bytes = 192 bits)
System.out.println("Reset Token: " + generateToken(24));
// Output: Reset Token: Mj7kL2... (32 characters, URL-safe)
// 6-digit OTP
System.out.println("OTP: " + generateOTP(6));
// Output: OTP: 384729
// NEVER use java.util.Random for security tokens!
// Random is predictable. SecureRandom uses OS entropy sources.
}
}
A common need is encrypting sensitive values in configuration files (database passwords, API keys) so they are not stored in plaintext:
import javax.crypto.Cipher;
import javax.crypto.SecretKey;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.SecretKeySpec;
import java.security.SecureRandom;
import java.util.Base64;
import java.util.HashMap;
import java.util.Map;
public class ConfigEncryptor {
private static final int GCM_IV_LENGTH = 12;
private static final int GCM_TAG_LENGTH = 128;
private static final String PREFIX = "ENC(";
private static final String SUFFIX = ")";
private final SecretKey key;
public ConfigEncryptor(String base64Key) {
byte[] keyBytes = Base64.getDecoder().decode(base64Key);
this.key = new SecretKeySpec(keyBytes, "AES");
}
public String encryptValue(String plaintext) throws Exception {
byte[] iv = new byte[GCM_IV_LENGTH];
new SecureRandom().nextBytes(iv);
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(GCM_TAG_LENGTH, iv));
byte[] ciphertext = cipher.doFinal(plaintext.getBytes("UTF-8"));
byte[] combined = new byte[iv.length + ciphertext.length];
System.arraycopy(iv, 0, combined, 0, iv.length);
System.arraycopy(ciphertext, 0, combined, iv.length, ciphertext.length);
return PREFIX + Base64.getEncoder().encodeToString(combined) + SUFFIX;
}
public String decryptValue(String encryptedValue) throws Exception {
// Strip ENC(...) wrapper
String base64 = encryptedValue.substring(PREFIX.length(), encryptedValue.length() - SUFFIX.length());
byte[] combined = Base64.getDecoder().decode(base64);
byte[] iv = new byte[GCM_IV_LENGTH];
System.arraycopy(combined, 0, iv, 0, iv.length);
byte[] ciphertext = new byte[combined.length - iv.length];
System.arraycopy(combined, iv.length, ciphertext, 0, ciphertext.length);
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, key, new GCMParameterSpec(GCM_TAG_LENGTH, iv));
return new String(cipher.doFinal(ciphertext), "UTF-8");
}
public boolean isEncrypted(String value) {
return value.startsWith(PREFIX) && value.endsWith(SUFFIX);
}
public static void main(String[] args) throws Exception {
// In production, this key comes from environment variable or key vault
// Generate once: Base64.getEncoder().encodeToString(KeyGenerator...generateKey().getEncoded())
String masterKey = "k7Hn3pRqSvWxYz1A2bCdEfGhIjKlMnOp4qRsTuVw5X8=";
ConfigEncryptor encryptor = new ConfigEncryptor(masterKey);
// Encrypt sensitive config values
Map config = new HashMap<>();
config.put("db.url", "jdbc:mysql://localhost:3306/mydb");
config.put("db.password", encryptor.encryptValue("SuperSecret123!"));
config.put("api.key", encryptor.encryptValue("sk-abc123def456"));
// Print config (safe to commit to version control)
System.out.println("=== Encrypted Config ===");
for (Map.Entry entry : config.entrySet()) {
System.out.println(entry.getKey() + " = " + entry.getValue());
}
// Output:
// db.url = jdbc:mysql://localhost:3306/mydb
// db.password = ENC(base64encodeddata...)
// api.key = ENC(base64encodeddata...)
// At runtime, decrypt the values
System.out.println("\n=== Decrypted at Runtime ===");
for (Map.Entry entry : config.entrySet()) {
String value = entry.getValue();
if (encryptor.isEncrypted(value)) {
value = encryptor.decryptValue(value);
}
System.out.println(entry.getKey() + " = " + value);
}
// Output:
// db.url = jdbc:mysql://localhost:3306/mydb
// db.password = SuperSecret123!
// api.key = sk-abc123def456
}
}
Cryptography is one of the easiest areas to get wrong in software development. A small mistake can make your encryption completely useless. Here are the most common mistakes Java developers make:
ECB (Electronic Codebook) mode encrypts each block independently. This means identical plaintext blocks produce identical ciphertext blocks, which leaks patterns in your data.
// WRONG -- ECB mode leaks patterns
Cipher cipher = Cipher.getInstance("AES/ECB/PKCS5Padding"); // NEVER DO THIS
cipher.init(Cipher.ENCRYPT_MODE, key);
// Encrypting "AAAAAAAAAAAAAAAA" twice produces the SAME ciphertext
// An attacker can see which blocks are identical
// RIGHT -- Use GCM mode (provides encryption + authentication)
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
byte[] iv = new byte[12];
new SecureRandom().nextBytes(iv);
cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(128, iv));
// WRONG -- key is visible in source code and version control
private static final String SECRET_KEY = "MyHardcodedKey123";
// WRONG -- key is in a config file committed to Git
// application.properties: encryption.key=MyHardcodedKey123
// RIGHT -- key comes from environment variable
String key = System.getenv("ENCRYPTION_KEY");
if (key == null) {
throw new IllegalStateException("ENCRYPTION_KEY environment variable not set");
}
// RIGHT -- key comes from a secrets manager (AWS Secrets Manager, HashiCorp Vault)
// String key = secretsManager.getSecret("encryption-key");
import java.util.Random; import java.security.SecureRandom; // WRONG -- java.util.Random is predictable (uses a linear congruential generator) // If an attacker knows the seed, they can predict ALL future values Random insecureRandom = new Random(); byte[] iv = new byte[16]; insecureRandom.nextBytes(iv); // PREDICTABLE IV -- defeats the purpose of encryption // RIGHT -- SecureRandom uses OS entropy (e.g., /dev/urandom on Linux) SecureRandom secureRandom = new SecureRandom(); byte[] safeIv = new byte[16]; secureRandom.nextBytes(safeIv); // Cryptographically unpredictable
// WRONG -- MD5 has known collision attacks since 2004
MessageDigest md5 = MessageDigest.getInstance("MD5");
byte[] hash = md5.digest(data.getBytes()); // Two different inputs can produce the same hash
// WRONG -- SHA-1 has been broken since 2017 (Google's SHAttered attack)
MessageDigest sha1 = MessageDigest.getInstance("SHA-1");
// RIGHT -- Use SHA-256 or SHA-512
MessageDigest sha256 = MessageDigest.getInstance("SHA-256");
// For passwords, do NOT use any SHA variant -- use bcrypt/scrypt/Argon2
// WRONG -- AES-CBC without HMAC. An attacker can modify the ciphertext
// and you will decrypt garbage without knowing it was tampered with.
Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
// ... encrypt ...
// No way to detect if ciphertext was modified!
// RIGHT -- AES-GCM provides authenticated encryption.
// If the ciphertext is modified, decryption throws AEADBadTagException.
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
// ... encrypt ...
// Decryption automatically verifies integrity via the authentication tag
// WRONG -- Using a fixed IV private static final byte[] FIXED_IV = new byte[12]; // all zeros // Same key + same IV + different plaintexts = complete security failure in GCM mode // WRONG -- Using a counter without proper management int counter = 0; byte[] iv = ByteBuffer.allocate(12).putInt(counter++).array(); // If the counter resets (app restart), you reuse IVs! // RIGHT -- Always generate a random IV for each encryption byte[] iv = new byte[12]; new SecureRandom().nextBytes(iv); // Fresh random IV every time
// WRONG -- Inventing your own "encryption" algorithm
public static String myEncrypt(String text) {
StringBuilder result = new StringBuilder();
for (char c : text.toCharArray()) {
result.append((char) (c + 3)); // Caesar cipher -- trivially broken
}
return result.toString();
}
// WRONG -- XOR "encryption" with a repeated key
public static byte[] xorEncrypt(byte[] data, byte[] key) {
byte[] result = new byte[data.length];
for (int i = 0; i < data.length; i++) {
result[i] = (byte) (data[i] ^ key[i % key.length]); // trivially broken
}
return result;
}
// RIGHT -- Use standard algorithms that have been analyzed by cryptographers
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
// AES has been analyzed by thousands of cryptographers since 1998
// and is approved for TOP SECRET government data
Here is a summary of what you should and should not do when implementing cryptography in Java:
| Category | Do | Do Not |
|---|---|---|
| Symmetric Encryption | Use AES-256-GCM | Use ECB mode, DES, 3DES, or Blowfish |
| Asymmetric Encryption | Use RSA-OAEP (2048+ bits) or ECDSA | Use RSA-PKCS1v1.5 or RSA less than 2048 bits |
| Password Hashing | Use bcrypt, scrypt, or Argon2 | Use SHA-256, MD5, or any fast hash for passwords |
| General Hashing | Use SHA-256 or SHA-512 | Use MD5 or SHA-1 for security purposes |
| Random Numbers | Use SecureRandom | Use java.util.Random for crypto |
| Key Storage | Use env vars, secrets manager, or key vault | Hardcode keys in source code or config files |
| IV / Nonce | Generate a fresh random IV for each encryption | Reuse IVs or use fixed/predictable IVs |
| Authentication | Use authenticated encryption (GCM) or HMAC | Encrypt without verifying integrity |
| Algorithm Design | Use standard, well-tested algorithms | Invent your own encryption scheme |
| Comparison | Use constant-time comparison for MACs/hashes | Use String.equals() for crypto comparisons |
| Key Size | AES-256, RSA-2048+, HMAC-SHA256 | AES-128 (acceptable), RSA-1024 (broken) |
| Dependencies | Keep crypto libraries updated | Use deprecated or unmaintained crypto libraries |
Additional guidance:
char[] instead of String for passwords (strings are immutable and linger in memory). Call Arrays.fill(charArray, '\0') when done.finally blocks to ensure keys and sensitive byte arrays are zeroed out.
Let us bring everything together in a practical SecureVault class that provides the four most common cryptographic operations: encrypt/decrypt strings, hash passwords, generate secure tokens, and verify data integrity with HMAC.
This class is designed to be production-quality. You can drop it into your project and use it immediately.
import javax.crypto.Cipher;
import javax.crypto.KeyGenerator;
import javax.crypto.Mac;
import javax.crypto.SecretKey;
import javax.crypto.SecretKeyFactory;
import javax.crypto.spec.GCMParameterSpec;
import javax.crypto.spec.PBEKeySpec;
import javax.crypto.spec.SecretKeySpec;
import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.SecureRandom;
import java.util.Arrays;
import java.util.Base64;
/**
* SecureVault -- a utility class for common cryptographic operations.
*
* Features:
* - AES-256-GCM encryption/decryption (authenticated encryption)
* - PBKDF2 password hashing with salt
* - HMAC-SHA256 message authentication
* - Cryptographically secure token generation
* - SHA-256 hashing for data integrity
*/
public class SecureVault {
private static final int GCM_IV_LENGTH = 12; // 96 bits
private static final int GCM_TAG_LENGTH = 128; // 128 bits
private static final int PBKDF2_ITERATIONS = 210_000;
private static final int PBKDF2_KEY_LENGTH = 256; // bits
private static final int SALT_LENGTH = 16; // bytes
private final SecureRandom secureRandom = new SecureRandom();
// ===== ENCRYPTION / DECRYPTION (AES-256-GCM) =====
/**
* Generate a new AES-256 encryption key.
*/
public SecretKey generateEncryptionKey() throws Exception {
KeyGenerator keyGen = KeyGenerator.getInstance("AES");
keyGen.init(256);
return keyGen.generateKey();
}
/**
* Reconstruct a key from its Base64-encoded form.
*/
public SecretKey keyFromBase64(String base64Key) {
byte[] keyBytes = Base64.getDecoder().decode(base64Key);
return new SecretKeySpec(keyBytes, "AES");
}
/**
* Export a key as a Base64 string (for secure storage).
*/
public String keyToBase64(SecretKey key) {
return Base64.getEncoder().encodeToString(key.getEncoded());
}
/**
* Encrypt a string using AES-256-GCM.
* The IV is prepended to the ciphertext in the output.
*/
public String encrypt(String plaintext, SecretKey key) throws Exception {
byte[] iv = new byte[GCM_IV_LENGTH];
secureRandom.nextBytes(iv);
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
cipher.init(Cipher.ENCRYPT_MODE, key, new GCMParameterSpec(GCM_TAG_LENGTH, iv));
byte[] ciphertext = cipher.doFinal(plaintext.getBytes(StandardCharsets.UTF_8));
byte[] combined = new byte[iv.length + ciphertext.length];
System.arraycopy(iv, 0, combined, 0, iv.length);
System.arraycopy(ciphertext, 0, combined, iv.length, ciphertext.length);
return Base64.getEncoder().encodeToString(combined);
}
/**
* Decrypt a string that was encrypted with encrypt().
* Throws AEADBadTagException if the ciphertext was tampered with.
*/
public String decrypt(String encryptedBase64, SecretKey key) throws Exception {
byte[] combined = Base64.getDecoder().decode(encryptedBase64);
byte[] iv = Arrays.copyOfRange(combined, 0, GCM_IV_LENGTH);
byte[] ciphertext = Arrays.copyOfRange(combined, GCM_IV_LENGTH, combined.length);
Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
cipher.init(Cipher.DECRYPT_MODE, key, new GCMParameterSpec(GCM_TAG_LENGTH, iv));
byte[] plaintext = cipher.doFinal(ciphertext);
return new String(plaintext, StandardCharsets.UTF_8);
}
// ===== PASSWORD HASHING (PBKDF2) =====
/**
* Hash a password using PBKDF2-HMAC-SHA256.
* Returns: "iterations:base64Salt:base64Hash"
*/
public String hashPassword(String password) throws Exception {
byte[] salt = new byte[SALT_LENGTH];
secureRandom.nextBytes(salt);
byte[] hash = pbkdf2(password.toCharArray(), salt, PBKDF2_ITERATIONS, PBKDF2_KEY_LENGTH);
return PBKDF2_ITERATIONS + ":"
+ Base64.getEncoder().encodeToString(salt) + ":"
+ Base64.getEncoder().encodeToString(hash);
}
/**
* Verify a password against a stored hash.
*/
public boolean verifyPassword(String password, String storedHash) throws Exception {
String[] parts = storedHash.split(":");
int iterations = Integer.parseInt(parts[0]);
byte[] salt = Base64.getDecoder().decode(parts[1]);
byte[] expectedHash = Base64.getDecoder().decode(parts[2]);
byte[] actualHash = pbkdf2(password.toCharArray(), salt, iterations, expectedHash.length * 8);
return MessageDigest.isEqual(expectedHash, actualHash);
}
private byte[] pbkdf2(char[] password, byte[] salt, int iterations, int keyLength)
throws Exception {
PBEKeySpec spec = new PBEKeySpec(password, salt, iterations, keyLength);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
return factory.generateSecret(spec).getEncoded();
}
// ===== HMAC (Message Authentication) =====
/**
* Generate an HMAC-SHA256 for data integrity verification.
*/
public String hmac(String data, SecretKey key) throws Exception {
Mac mac = Mac.getInstance("HmacSHA256");
mac.init(key);
byte[] hmacBytes = mac.doFinal(data.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(hmacBytes);
}
/**
* Verify an HMAC using constant-time comparison.
*/
public boolean verifyHmac(String data, String expectedHmac, SecretKey key) throws Exception {
String actualHmac = hmac(data, key);
return MessageDigest.isEqual(
actualHmac.getBytes(StandardCharsets.UTF_8),
expectedHmac.getBytes(StandardCharsets.UTF_8)
);
}
// ===== HASHING (SHA-256) =====
/**
* Hash data using SHA-256 (for data integrity, NOT passwords).
*/
public String sha256(String data) throws Exception {
MessageDigest digest = MessageDigest.getInstance("SHA-256");
byte[] hashBytes = digest.digest(data.getBytes(StandardCharsets.UTF_8));
return Base64.getEncoder().encodeToString(hashBytes);
}
// ===== TOKEN GENERATION =====
/**
* Generate a URL-safe random token.
*/
public String generateToken(int byteLength) {
byte[] tokenBytes = new byte[byteLength];
secureRandom.nextBytes(tokenBytes);
return Base64.getUrlEncoder().withoutPadding().encodeToString(tokenBytes);
}
/**
* Generate a numeric OTP.
*/
public String generateOTP(int digits) {
int max = (int) Math.pow(10, digits);
int otp = secureRandom.nextInt(max);
return String.format("%0" + digits + "d", otp);
}
}
Now let us use SecureVault in a complete application that demonstrates every feature:
import javax.crypto.SecretKey;
import javax.crypto.spec.SecretKeySpec;
public class SecureVaultDemo {
public static void main(String[] args) throws Exception {
SecureVault vault = new SecureVault();
System.out.println("===== SecureVault Demo =====\n");
// ---- 1. Encryption / Decryption ----
System.out.println("--- 1. AES-256-GCM Encryption ---");
SecretKey encKey = vault.generateEncryptionKey();
String keyBase64 = vault.keyToBase64(encKey);
System.out.println("Key (Base64): " + keyBase64);
String secret = "My credit card number is 4111-1111-1111-1111";
String encrypted = vault.encrypt(secret, encKey);
System.out.println("Plaintext: " + secret);
System.out.println("Encrypted: " + encrypted);
String decrypted = vault.decrypt(encrypted, encKey);
System.out.println("Decrypted: " + decrypted);
System.out.println("Match: " + secret.equals(decrypted));
// Same plaintext, different ciphertext (random IV)
String encrypted2 = vault.encrypt(secret, encKey);
System.out.println("Same ciphertext? " + encrypted.equals(encrypted2));
// Output: Same ciphertext? false
// Reconstruct key from Base64
SecretKey restoredKey = vault.keyFromBase64(keyBase64);
String decrypted2 = vault.decrypt(encrypted, restoredKey);
System.out.println("Decrypted with restored key: " + decrypted2);
// ---- 2. Password Hashing ----
System.out.println("\n--- 2. Password Hashing (PBKDF2) ---");
String password = "S3cur3P@ssw0rd!";
String hashedPassword = vault.hashPassword(password);
System.out.println("Password: " + password);
System.out.println("Hashed: " + hashedPassword);
boolean correctPassword = vault.verifyPassword(password, hashedPassword);
System.out.println("Correct password: " + correctPassword);
// Output: Correct password: true
boolean wrongPassword = vault.verifyPassword("wrongpassword", hashedPassword);
System.out.println("Wrong password: " + wrongPassword);
// Output: Wrong password: false
// ---- 3. HMAC ----
System.out.println("\n--- 3. HMAC-SHA256 ---");
SecretKey hmacKey = vault.generateEncryptionKey(); // can reuse AES key for HMAC
String apiPayload = "{\"orderId\":\"12345\",\"amount\":99.99}";
String signature = vault.hmac(apiPayload, hmacKey);
System.out.println("Payload: " + apiPayload);
System.out.println("HMAC: " + signature);
boolean validHmac = vault.verifyHmac(apiPayload, signature, hmacKey);
System.out.println("Valid HMAC: " + validHmac);
// Output: Valid HMAC: true
String tamperedPayload = "{\"orderId\":\"12345\",\"amount\":0.01}";
boolean tamperedHmac = vault.verifyHmac(tamperedPayload, signature, hmacKey);
System.out.println("Tampered HMAC valid: " + tamperedHmac);
// Output: Tampered HMAC valid: false
// ---- 4. SHA-256 Hashing ----
System.out.println("\n--- 4. SHA-256 Hashing ---");
String fileContent = "Important document content";
String hash = vault.sha256(fileContent);
System.out.println("Content: " + fileContent);
System.out.println("SHA-256: " + hash);
// Verify integrity
String hash2 = vault.sha256(fileContent);
System.out.println("Integrity check: " + hash.equals(hash2));
// Output: Integrity check: true
// ---- 5. Token Generation ----
System.out.println("\n--- 5. Secure Token Generation ---");
System.out.println("API Key (32 bytes): " + vault.generateToken(32));
System.out.println("Session Token (16 bytes): " + vault.generateToken(16));
System.out.println("Reset Token (24 bytes): " + vault.generateToken(24));
System.out.println("6-digit OTP: " + vault.generateOTP(6));
System.out.println("4-digit OTP: " + vault.generateOTP(4));
// ---- 6. Tamper Detection ----
System.out.println("\n--- 6. Tamper Detection (GCM) ---");
String sensitiveData = "Transfer $10,000 to account 98765";
String encryptedData = vault.encrypt(sensitiveData, encKey);
System.out.println("Encrypted: " + encryptedData);
try {
// Simulate tampering: modify one character in the Base64 string
char[] chars = encryptedData.toCharArray();
chars[20] = (chars[20] == 'A') ? 'B' : 'A'; // flip a character
String tampered = new String(chars);
vault.decrypt(tampered, encKey); // This should throw an exception
System.out.println("ERROR: Tampered data was accepted!");
} catch (Exception e) {
System.out.println("Tamper detected: " + e.getClass().getSimpleName());
// Output: Tamper detected: AEADBadTagException
}
System.out.println("\n===== Demo Complete =====");
}
}
The output of the complete demo:
===== SecureVault Demo =====
--- 1. AES-256-GCM Encryption ---
Key (Base64): kR7xN9pQ2sVwYz1A3bCdEfGhIjKlMnOp4qRsTuVw5X8=
Plaintext: My credit card number is 4111-1111-1111-1111
Encrypted: (Base64 encoded ciphertext -- different each time)
Decrypted: My credit card number is 4111-1111-1111-1111
Match: true
Same ciphertext? false
Decrypted with restored key: My credit card number is 4111-1111-1111-1111
--- 2. Password Hashing (PBKDF2) ---
Password: S3cur3P@ssw0rd!
Hashed: 210000:BASE64_SALT:BASE64_HASH
Correct password: true
Wrong password: false
--- 3. HMAC-SHA256 ---
Payload: {"orderId":"12345","amount":99.99}
HMAC: (Base64 encoded HMAC)
Valid HMAC: true
Tampered HMAC valid: false
--- 4. SHA-256 Hashing ---
Content: Important document content
SHA-256: (Base64 encoded hash)
Integrity check: true
--- 5. Secure Token Generation ---
API Key (32 bytes): xK9mPqR7sVwYz1A3bCdEfGhIjKlMnOp4qRsTuVw5X8
Session Token (16 bytes): a1B2c3D4e5F6g7H8i9J0kL
Reset Token (24 bytes): Mj7kL2nP9qR3sT5uV8wX0yZ1a2B3c4D
6-digit OTP: 384729
4-digit OTP: 9152
--- 6. Tamper Detection (GCM) ---
Encrypted: (Base64 encoded ciphertext)
Tamper detected: AEADBadTagException
===== Demo Complete =====
Here is a summary of the concepts covered in this tutorial and where each appears in the SecureVault example:
| # | Concept | Algorithm / Class | Where Used |
|---|---|---|---|
| 1 | Symmetric Encryption | AES-256-GCM | encrypt(), decrypt() |
| 2 | Key Generation | KeyGenerator | generateEncryptionKey() |
| 3 | Key Serialization | Base64, SecretKeySpec | keyToBase64(), keyFromBase64() |
| 4 | Initialization Vector | SecureRandom, GCMParameterSpec | Random IV in encrypt() |
| 5 | Authenticated Encryption | GCM authentication tag | Tamper detection in demo section 6 |
| 6 | Password Hashing | PBKDF2-HMAC-SHA256 | hashPassword(), verifyPassword() |
| 7 | Salt | SecureRandom | Random salt in hashPassword() |
| 8 | HMAC | HmacSHA256, Mac | hmac(), verifyHmac() |
| 9 | Constant-Time Comparison | MessageDigest.isEqual() | verifyPassword(), verifyHmac() |
| 10 | SHA-256 Hashing | MessageDigest | sha256() |
| 11 | Secure Token Generation | SecureRandom | generateToken(), generateOTP() |
| 12 | Base64 Encoding | Base64 URL-safe encoder | Throughout (key storage, ciphertext, tokens) |
| Task | Recommended Approach | Java Classes |
|---|---|---|
| Encrypt data | AES-256-GCM | Cipher, KeyGenerator, GCMParameterSpec, SecureRandom |
| Hash passwords | bcrypt or PBKDF2 | BCryptPasswordEncoder or SecretKeyFactory + PBEKeySpec |
| Verify data integrity | HMAC-SHA256 | Mac, SecretKeySpec |
| Hash data (non-password) | SHA-256 | MessageDigest |
| Generate random tokens | SecureRandom + Base64 | SecureRandom, Base64.getUrlEncoder() |
| Exchange keys securely | RSA-OAEP or ECDH | KeyPairGenerator, Cipher |
| Sign data | SHA256withRSA | Signature, KeyPairGenerator |
| Encode binary as text | Base64 | Base64.getEncoder() / getUrlEncoder() |
Almost every real-world application needs to persist data, and relational databases remain the backbone of most production systems. MySQL, the world’s most popular open-source relational database, pairs naturally with Python — one of the world’s most popular programming languages. Whether you are building a web application with Flask or Django, automating data pipelines, or writing microservices, knowing how to talk to MySQL from Python is a non-negotiable skill.
In this tutorial you will learn everything from establishing a basic connection to managing transactions, pooling connections for production workloads, and even mapping your tables to Python objects with SQLAlchemy. Every example is production-minded: parameterized queries, proper error handling, and clean resource management from the start.
The most common MySQL driver for Python is mysql-connector-python, maintained by Oracle. Install it with pip:
pip install mysql-connector-python
A popular alternative is PyMySQL, a pure-Python driver that requires no C extensions:
pip install pymysql
Both libraries follow the Python DB-API 2.0 specification (PEP 249), so the core patterns — connect, cursor, execute, fetch — are nearly identical. This tutorial uses mysql-connector-python for all examples. If you are using PyMySQL, swap the import and connection call and the rest of your code stays the same.
You will also need a running MySQL server. If you do not have one, the quickest path is Docker:
# Pull and run MySQL 8 in a container docker run --name mysql-tutorial \ -e MYSQL_ROOT_PASSWORD=rootpass \ -p 3306:3306 \ -d mysql:8
Every interaction starts with a connection. You provide the host, port, user, password, and optionally a database name:
import mysql.connector
# Establish a connection
conn = mysql.connector.connect(
host="127.0.0.1",
port=3306,
user="root",
password="rootpass"
)
print("Connected:", conn.is_connected()) # True
# Always close when done
conn.close()
If the connection fails — wrong password, server not running, network issue — mysql.connector.Error is raised. Always wrap your connection logic in a try/except block:
import mysql.connector
from mysql.connector import Error
try:
conn = mysql.connector.connect(
host="127.0.0.1",
user="root",
password="rootpass"
)
if conn.is_connected():
info = conn.get_server_info()
print(f"Connected to MySQL Server version {info}")
except Error as e:
print(f"Error connecting to MySQL: {e}")
finally:
if 'conn' in locals() and conn.is_connected():
conn.close()
print("Connection closed")
Once connected, use a cursor to execute SQL statements. Let us create a database and a table:
import mysql.connector
from mysql.connector import Error
conn = mysql.connector.connect(
host="127.0.0.1",
user="root",
password="rootpass"
)
cursor = conn.cursor()
# Create database
cursor.execute("CREATE DATABASE IF NOT EXISTS tutorial_db")
cursor.execute("USE tutorial_db")
# Create table
create_table_sql = """
CREATE TABLE IF NOT EXISTS users (
id INT AUTO_INCREMENT PRIMARY KEY,
username VARCHAR(50) NOT NULL UNIQUE,
email VARCHAR(100) NOT NULL,
age INT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
"""
cursor.execute(create_table_sql)
print("Database and table created successfully")
cursor.close()
conn.close()
You can also connect directly to a database by passing the database parameter:
conn = mysql.connector.connect(
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
CRUD — Create, Read, Update, Delete — covers the four fundamental data operations. Let us walk through each one.
Single insert:
import mysql.connector
conn = mysql.connector.connect(
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
cursor = conn.cursor()
sql = "INSERT INTO users (username, email, age) VALUES (%s, %s, %s)"
values = ("alice", "alice@example.com", 30)
cursor.execute(sql, values)
conn.commit() # IMPORTANT: commit the transaction
print(f"Inserted user with ID: {cursor.lastrowid}")
cursor.close()
conn.close()
Batch insert with executemany():
sql = "INSERT INTO users (username, email, age) VALUES (%s, %s, %s)"
users = [
("bob", "bob@example.com", 25),
("charlie", "charlie@example.com", 35),
("diana", "diana@example.com", 28),
("eve", "eve@example.com", 32),
]
cursor.executemany(sql, users)
conn.commit()
print(f"Inserted {cursor.rowcount} rows")
executemany() is significantly faster than looping with individual execute() calls because the driver can optimize the network round-trips.
The cursor provides three fetch methods:
fetchone() — returns the next row as a tuple, or Nonefetchall() — returns all remaining rows as a list of tuplesfetchmany(size) — returns up to size rows# Fetch all users
cursor.execute("SELECT id, username, email, age FROM users")
rows = cursor.fetchall()
for row in rows:
print(f"ID: {row[0]}, Username: {row[1]}, Email: {row[2]}, Age: {row[3]}")
For more readable code, use a dictionary cursor so each row is a dict instead of a tuple:
cursor = conn.cursor(dictionary=True)
cursor.execute("SELECT * FROM users WHERE age > %s", (28,))
for user in cursor.fetchall():
print(f"{user['username']} ({user['email']}) - Age {user['age']}")
Fetching one row at a time is memory-efficient for large result sets:
cursor.execute("SELECT * FROM users ORDER BY created_at DESC")
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
Fetching in batches balances memory and performance:
cursor.execute("SELECT * FROM users")
while True:
batch = cursor.fetchmany(size=2)
if not batch:
break
for row in batch:
print(row)
sql = "UPDATE users SET email = %s, age = %s WHERE username = %s"
values = ("alice_new@example.com", 31, "alice")
cursor.execute(sql, values)
conn.commit()
print(f"Rows affected: {cursor.rowcount}")
sql = "DELETE FROM users WHERE username = %s"
cursor.execute(sql, ("eve",))
conn.commit()
print(f"Deleted {cursor.rowcount} row(s)")
Always check cursor.rowcount after UPDATE and DELETE to confirm the operation affected the expected number of rows.
This is not optional — it is a hard requirement for any production code. Parameterized queries prevent SQL injection, one of the most dangerous and most common web vulnerabilities.
Never do this:
# DANGEROUS — SQL injection vulnerability!
username = input("Enter username: ")
cursor.execute(f"SELECT * FROM users WHERE username = '{username}'")
If a user enters ' OR '1'='1, that query returns every row in the table. Worse, they could enter '; DROP TABLE users; -- and destroy your data.
Always do this:
# SAFE — parameterized query
username = input("Enter username: ")
cursor.execute("SELECT * FROM users WHERE username = %s", (username,))
user = cursor.fetchone()
The %s placeholder tells the driver to properly escape and quote the value. This works regardless of what the user types — the database sees it as a literal value, not executable SQL.
Key rules:
%s as the placeholder (not ? — that is for SQLite)(value,)f"", .format(), %) to build SQLA transaction groups multiple SQL statements into a single atomic unit. Either all of them succeed, or none of them do. MySQL with InnoDB supports full ACID transactions.
import mysql.connector
from mysql.connector import Error
conn = mysql.connector.connect(
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
try:
cursor = conn.cursor()
# Transfer "credits" from alice to bob (both must succeed)
cursor.execute(
"UPDATE users SET age = age - 1 WHERE username = %s", ("alice",)
)
cursor.execute(
"UPDATE users SET age = age + 1 WHERE username = %s", ("bob",)
)
conn.commit() # Both updates are saved
print("Transaction committed")
except Error as e:
conn.rollback() # Undo everything if any statement fails
print(f"Transaction rolled back: {e}")
finally:
cursor.close()
conn.close()
By default, mysql-connector-python does not auto-commit. You must call conn.commit() explicitly. If you want auto-commit behavior (not recommended for multi-statement operations), set it at connection time:
# Auto-commit mode — each statement is its own transaction
conn = mysql.connector.connect(
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db",
autocommit=True
)
When to use explicit transactions:
Opening and closing database connections is expensive. In a web application handling hundreds of requests per second, creating a new connection for every request wastes time and resources. Connection pooling solves this by maintaining a pool of reusable connections.
from mysql.connector import pooling
# Create a connection pool
pool = pooling.MySQLConnectionPool(
pool_name="tutorial_pool",
pool_size=5,
pool_reset_session=True,
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
# Get a connection from the pool
conn = pool.get_connection()
cursor = conn.cursor(dictionary=True)
cursor.execute("SELECT * FROM users")
for user in cursor.fetchall():
print(user)
cursor.close()
conn.close() # Returns the connection to the pool, does not destroy it
When you call conn.close() on a pooled connection, it goes back to the pool instead of being destroyed. The next call to pool.get_connection() can reuse it immediately.
Pool sizing guidelines:
pool_size=5 and increase based on load testingSHOW STATUS LIKE 'Threads_connected' in MySQLHere is a thread-safe pattern for a web application:
from mysql.connector import pooling, Error
class Database:
"""Thread-safe database access using connection pooling."""
def __init__(self, **kwargs):
self.pool = pooling.MySQLConnectionPool(
pool_name="app_pool",
pool_size=10,
**kwargs
)
def execute_query(self, query, params=None, fetch=False):
conn = self.pool.get_connection()
try:
cursor = conn.cursor(dictionary=True)
cursor.execute(query, params)
if fetch:
result = cursor.fetchall()
else:
conn.commit()
result = cursor.rowcount
return result
except Error as e:
conn.rollback()
raise e
finally:
cursor.close()
conn.close()
# Usage
db = Database(
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
users = db.execute_query("SELECT * FROM users WHERE age > %s", (25,), fetch=True)
print(users)
Context managers (the with statement) guarantee that resources are cleaned up even if an exception occurs. Let us build a reusable context manager for database operations:
from contextlib import contextmanager
import mysql.connector
from mysql.connector import Error
@contextmanager
def get_db_connection(config):
"""Context manager that provides a database connection."""
conn = mysql.connector.connect(**config)
try:
yield conn
except Error as e:
conn.rollback()
raise e
finally:
conn.close()
@contextmanager
def get_db_cursor(conn, dictionary=True):
"""Context manager that provides a cursor and commits on success."""
cursor = conn.cursor(dictionary=dictionary)
try:
yield cursor
conn.commit()
except Error as e:
conn.rollback()
raise e
finally:
cursor.close()
# Configuration
DB_CONFIG = {
"host": "127.0.0.1",
"user": "root",
"password": "rootpass",
"database": "tutorial_db"
}
# Usage — clean and exception-safe
with get_db_connection(DB_CONFIG) as conn:
with get_db_cursor(conn) as cursor:
cursor.execute(
"INSERT INTO users (username, email, age) VALUES (%s, %s, %s)",
("frank", "frank@example.com", 29)
)
print(f"Inserted row ID: {cursor.lastrowid}")
# Connection and cursor are automatically closed here
This pattern is the recommended way to manage database resources in production Python applications. It eliminates an entire class of bugs — leaked connections, uncommitted transactions, and unclosed cursors.
For pooled connections, combine the two patterns:
from mysql.connector import pooling
from contextlib import contextmanager
pool = pooling.MySQLConnectionPool(
pool_name="app_pool",
pool_size=5,
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
@contextmanager
def get_connection():
conn = pool.get_connection()
try:
yield conn
finally:
conn.close() # Returns to pool
@contextmanager
def get_cursor(conn):
cursor = conn.cursor(dictionary=True)
try:
yield cursor
conn.commit()
except Exception:
conn.rollback()
raise
finally:
cursor.close()
# Usage
with get_connection() as conn:
with get_cursor(conn) as cursor:
cursor.execute("SELECT COUNT(*) AS total FROM users")
result = cursor.fetchone()
print(f"Total users: {result['total']}")
So far, every example has used raw SQL. That works well for simple applications and gives you full control. But as your application grows — more tables, more relationships, more complex queries — writing raw SQL becomes tedious and error-prone. That is where an ORM (Object-Relational Mapper) shines.
SQLAlchemy is Python’s most powerful and most widely used ORM. Install it alongside the MySQL driver:
pip install sqlalchemy mysql-connector-python
SQLAlchemy needs three things to get started:
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, DeclarativeBase
# Connection URL format: mysql+connector://user:password@host:port/database
engine = create_engine(
"mysql+mysqlconnector://root:rootpass@127.0.0.1:3306/tutorial_db",
echo=False, # Set True to log all SQL statements
pool_size=5,
max_overflow=10
)
# Create a session factory
SessionLocal = sessionmaker(bind=engine)
# Base class for models
class Base(DeclarativeBase):
pass
Each model class maps to a database table. Columns become class attributes:
from sqlalchemy import Column, Integer, String, DateTime, ForeignKey
from sqlalchemy.orm import relationship
from sqlalchemy.sql import func
class User(Base):
__tablename__ = "orm_users"
id = Column(Integer, primary_key=True, autoincrement=True)
username = Column(String(50), unique=True, nullable=False)
email = Column(String(100), nullable=False)
age = Column(Integer)
created_at = Column(DateTime, server_default=func.now())
# One-to-many relationship
posts = relationship("Post", back_populates="author",
cascade="all, delete-orphan")
def __repr__(self):
return f"<User(id={self.id}, username='{self.username}')>"
class Post(Base):
__tablename__ = "orm_posts"
id = Column(Integer, primary_key=True, autoincrement=True)
title = Column(String(200), nullable=False)
body = Column(String(5000))
user_id = Column(Integer, ForeignKey("orm_users.id"), nullable=False)
created_at = Column(DateTime, server_default=func.now())
author = relationship("User", back_populates="posts")
def __repr__(self):
return f"<Post(id={self.id}, title='{self.title}')>"
# Create all tables
Base.metadata.create_all(engine)
# CREATE
session = SessionLocal()
new_user = User(username="grace", email="grace@example.com", age=27)
session.add(new_user)
session.commit()
print(f"Created: {new_user}")
# Add a post for this user
new_post = Post(title="My First Post", body="Hello from SQLAlchemy!",
user_id=new_user.id)
session.add(new_post)
session.commit()
# READ
user = session.query(User).filter_by(username="grace").first()
print(f"Found: {user}")
print(f"Posts: {user.posts}") # Lazy-loaded relationship
# All users older than 25
young_users = session.query(User).filter(User.age > 25).all()
for u in young_users:
print(u)
# UPDATE
user.email = "grace_updated@example.com"
session.commit()
# DELETE
session.delete(user) # Also deletes posts due to cascade
session.commit()
session.close()
With the ORM, notice how you never write a single line of SQL. SQLAlchemy generates it for you, handles parameterization, and manages the transaction lifecycle.
from contextlib import contextmanager
@contextmanager
def get_session():
session = SessionLocal()
try:
yield session
session.commit()
except Exception:
session.rollback()
raise
finally:
session.close()
# Usage
with get_session() as session:
user = User(username="henry", email="henry@example.com", age=33)
session.add(user)
# Automatically committed when the block exits without error
| Use ORM When | Use Raw SQL When |
|---|---|
| Building a CRUD-heavy application | Running complex analytical queries |
| You need relationship management | You need maximum query performance |
| Rapid prototyping and iteration | Migrating or bulk-loading data |
| Working with multiple database backends | Using database-specific features |
| Team members vary in SQL skill | Debugging performance issues |
Many production applications use both — ORM for standard CRUD and raw SQL (via session.execute()) for complex queries and reporting.
A complete user management module with registration, authentication, and profile updates:
import mysql.connector
from mysql.connector import pooling, Error
import hashlib
import os
from contextlib import contextmanager
# --- Database Setup ---
pool = pooling.MySQLConnectionPool(
pool_name="user_mgmt_pool",
pool_size=5,
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
@contextmanager
def get_connection():
conn = pool.get_connection()
try:
yield conn
finally:
conn.close()
@contextmanager
def get_cursor(conn, dictionary=True):
cursor = conn.cursor(dictionary=dictionary)
try:
yield cursor
conn.commit()
except Error:
conn.rollback()
raise
finally:
cursor.close()
def init_db():
"""Create the accounts table if it does not exist."""
with get_connection() as conn:
with get_cursor(conn) as cursor:
cursor.execute("""
CREATE TABLE IF NOT EXISTS accounts (
id INT AUTO_INCREMENT PRIMARY KEY,
username VARCHAR(50) NOT NULL UNIQUE,
email VARCHAR(100) NOT NULL UNIQUE,
password_hash VARCHAR(128) NOT NULL,
salt VARCHAR(64) NOT NULL,
full_name VARCHAR(100),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
ON UPDATE CURRENT_TIMESTAMP
)
""")
def hash_password(password, salt=None):
"""Hash a password with a random salt."""
if salt is None:
salt = os.urandom(32).hex()
hashed = hashlib.sha256((salt + password).encode()).hexdigest()
return hashed, salt
def register_user(username, email, password, full_name=None):
"""Register a new user. Returns user ID on success."""
password_hash, salt = hash_password(password)
with get_connection() as conn:
with get_cursor(conn) as cursor:
try:
cursor.execute(
"""INSERT INTO accounts
(username, email, password_hash, salt, full_name)
VALUES (%s, %s, %s, %s, %s)""",
(username, email, password_hash, salt, full_name)
)
print(f"User '{username}' registered with ID {cursor.lastrowid}")
return cursor.lastrowid
except Error as e:
if e.errno == 1062: # Duplicate entry
print("Registration failed: username or email already exists")
return None
raise
def login(username, password):
"""Authenticate a user. Returns user dict or None."""
with get_connection() as conn:
with get_cursor(conn) as cursor:
cursor.execute(
"""SELECT id, username, email, password_hash, salt, full_name
FROM accounts WHERE username = %s""",
(username,)
)
user = cursor.fetchone()
if user is None:
print("Login failed: user not found")
return None
hashed, _ = hash_password(password, user["salt"])
if hashed != user["password_hash"]:
print("Login failed: incorrect password")
return None
print(f"Welcome back, {user['full_name'] or user['username']}!")
return {
"id": user["id"],
"username": user["username"],
"email": user["email"],
"full_name": user["full_name"]
}
def update_profile(user_id, **kwargs):
"""Update user profile fields. Only updates provided fields."""
allowed_fields = {"email", "full_name"}
updates = {k: v for k, v in kwargs.items() if k in allowed_fields}
if not updates:
print("No valid fields to update")
return False
set_clause = ", ".join(f"{field} = %s" for field in updates)
values = list(updates.values()) + [user_id]
with get_connection() as conn:
with get_cursor(conn) as cursor:
cursor.execute(
f"UPDATE accounts SET {set_clause} WHERE id = %s",
tuple(values)
)
if cursor.rowcount > 0:
print(f"Profile updated for user ID {user_id}")
return True
print("User not found")
return False
# --- Demo ---
if __name__ == "__main__":
init_db()
# Register
user_id = register_user(
"johndoe", "john@example.com", "s3cur3P@ss", "John Doe"
)
# Login
user = login("johndoe", "s3cur3P@ss")
# Update profile
if user:
update_profile(
user["id"],
email="john.doe@newmail.com",
full_name="John A. Doe"
)
import mysql.connector
from mysql.connector import pooling, Error
from contextlib import contextmanager
from decimal import Decimal
pool = pooling.MySQLConnectionPool(
pool_name="inventory_pool",
pool_size=5,
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
@contextmanager
def db_cursor(dictionary=True):
conn = pool.get_connection()
cursor = conn.cursor(dictionary=dictionary)
try:
yield cursor
conn.commit()
except Error:
conn.rollback()
raise
finally:
cursor.close()
conn.close()
def init_inventory():
with db_cursor() as cursor:
cursor.execute("""
CREATE TABLE IF NOT EXISTS products (
id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(100) NOT NULL,
sku VARCHAR(50) NOT NULL UNIQUE,
price DECIMAL(10, 2) NOT NULL,
quantity INT NOT NULL DEFAULT 0,
category VARCHAR(50),
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
def add_product(name, sku, price, quantity=0, category=None):
with db_cursor() as cursor:
cursor.execute(
"""INSERT INTO products (name, sku, price, quantity, category)
VALUES (%s, %s, %s, %s, %s)""",
(name, sku, price, quantity, category)
)
return cursor.lastrowid
def restock(sku, amount):
"""Add stock to an existing product."""
with db_cursor() as cursor:
cursor.execute(
"UPDATE products SET quantity = quantity + %s WHERE sku = %s",
(amount, sku)
)
if cursor.rowcount == 0:
raise ValueError(f"Product with SKU '{sku}' not found")
print(f"Restocked {amount} units of {sku}")
def sell(sku, amount):
"""Reduce stock. Raises error if insufficient stock."""
with db_cursor() as cursor:
# Check current stock
cursor.execute(
"SELECT quantity FROM products WHERE sku = %s", (sku,)
)
product = cursor.fetchone()
if product is None:
raise ValueError(f"Product '{sku}' not found")
if product["quantity"] < amount:
raise ValueError(
f"Insufficient stock: {product['quantity']} available, "
f"{amount} requested"
)
cursor.execute(
"UPDATE products SET quantity = quantity - %s WHERE sku = %s",
(amount, sku)
)
print(f"Sold {amount} units of {sku}")
def get_low_stock(threshold=10):
"""Find products that need restocking."""
with db_cursor() as cursor:
cursor.execute(
"""SELECT name, sku, quantity FROM products
WHERE quantity <= %s ORDER BY quantity ASC""",
(threshold,)
)
return cursor.fetchall()
def get_inventory_value():
"""Calculate total inventory value."""
with db_cursor() as cursor:
cursor.execute(
"SELECT SUM(price * quantity) AS total_value FROM products"
)
result = cursor.fetchone()
return result["total_value"] or Decimal("0.00")
def search_products(keyword):
"""Search products by name or category."""
with db_cursor() as cursor:
pattern = f"%{keyword}%"
cursor.execute(
"""SELECT * FROM products
WHERE name LIKE %s OR category LIKE %s""",
(pattern, pattern)
)
return cursor.fetchall()
# --- Demo ---
if __name__ == "__main__":
init_inventory()
# Add products
add_product("Mechanical Keyboard", "KB-001", 89.99, 50, "Electronics")
add_product("USB-C Cable", "CB-001", 12.99, 200, "Accessories")
add_product("Monitor Stand", "MS-001", 45.00, 15, "Furniture")
add_product("Webcam HD", "WC-001", 59.99, 8, "Electronics")
# Sell some items
sell("KB-001", 5)
restock("WC-001", 20)
# Reports
print("\nLow stock items:")
for item in get_low_stock(threshold=20):
print(f" {item['name']} (SKU: {item['sku']}): {item['quantity']} left")
print(f"\nTotal inventory value: ${get_inventory_value():,.2f}")
print("\nElectronics products:")
for p in search_products("Electronics"):
print(f" {p['name']} - ${p['price']} ({p['quantity']} in stock)")
A reusable data access layer that any application can build on — similar to a repository pattern used in web frameworks:
import mysql.connector
from mysql.connector import pooling, Error
from contextlib import contextmanager
class DataAccessLayer:
"""A generic, reusable data access layer for MySQL."""
def __init__(self, host, user, password, database, pool_size=5):
self.pool = pooling.MySQLConnectionPool(
pool_name="dal_pool",
pool_size=pool_size,
host=host,
user=user,
password=password,
database=database
)
@contextmanager
def _get_cursor(self):
conn = self.pool.get_connection()
cursor = conn.cursor(dictionary=True)
try:
yield cursor, conn
finally:
cursor.close()
conn.close()
def fetch_all(self, query, params=None):
"""Execute a SELECT and return all rows."""
with self._get_cursor() as (cursor, conn):
cursor.execute(query, params)
return cursor.fetchall()
def fetch_one(self, query, params=None):
"""Execute a SELECT and return the first row."""
with self._get_cursor() as (cursor, conn):
cursor.execute(query, params)
return cursor.fetchone()
def execute(self, query, params=None):
"""Execute INSERT, UPDATE, or DELETE. Returns affected row count."""
with self._get_cursor() as (cursor, conn):
cursor.execute(query, params)
conn.commit()
return cursor.rowcount
def insert(self, query, params=None):
"""Execute an INSERT and return the new row's ID."""
with self._get_cursor() as (cursor, conn):
cursor.execute(query, params)
conn.commit()
return cursor.lastrowid
def execute_many(self, query, params_list):
"""Execute a batch operation. Returns affected row count."""
with self._get_cursor() as (cursor, conn):
cursor.executemany(query, params_list)
conn.commit()
return cursor.rowcount
def execute_transaction(self, operations):
"""
Execute multiple operations in a single transaction.
operations: list of (query, params) tuples
"""
with self._get_cursor() as (cursor, conn):
try:
for query, params in operations:
cursor.execute(query, params)
conn.commit()
return True
except Error:
conn.rollback()
raise
# --- Usage Example ---
dal = DataAccessLayer(
host="127.0.0.1",
user="root",
password="rootpass",
database="tutorial_db"
)
# Insert
user_id = dal.insert(
"INSERT INTO users (username, email, age) VALUES (%s, %s, %s)",
("ivy", "ivy@example.com", 26)
)
# Read
users = dal.fetch_all("SELECT * FROM users WHERE age > %s", (25,))
for user in users:
print(user)
# Update
affected = dal.execute(
"UPDATE users SET age = %s WHERE username = %s",
(27, "ivy")
)
# Transaction
dal.execute_transaction([
("UPDATE users SET age = age - 1 WHERE username = %s", ("alice",)),
("UPDATE users SET age = age + 1 WHERE username = %s", ("bob",)),
])
These are the mistakes that burn developers most often. Learn them here so you do not learn them in a production outage.
We covered this above, but it bears repeating. Never build SQL strings with user input. Always use parameterized queries. This is the number-one security vulnerability in web applications, and it is completely preventable.
If your INSERTs and UPDATEs seem to work but the data disappears, you forgot to call conn.commit(). The default mode is manual commit — every write must be explicitly committed.
# This does NOTHING to the database without commit()
cursor.execute(
"INSERT INTO users (username, email) VALUES (%s, %s)",
("ghost", "ghost@example.com")
)
# conn.commit() <-- Missing! Data is lost when connection closes.
If you open connections without closing them, your application eventually exhausts the MySQL connection limit (default: 151). Use context managers or try/finally blocks to guarantee cleanup:
# BAD — if an exception occurs, connection is never closed
conn = mysql.connector.connect(**config)
cursor = conn.cursor()
cursor.execute("SELECT * FROM users")
# ... exception here means conn.close() never runs
conn.close()
# GOOD — finally block guarantees cleanup
conn = mysql.connector.connect(**config)
try:
cursor = conn.cursor()
cursor.execute("SELECT * FROM users")
results = cursor.fetchall()
finally:
conn.close()
This is especially common with ORMs. If you load a list of users, then loop through them loading each user's posts individually, you make 1 + N queries instead of a single JOIN:
# BAD — N+1 queries
users = session.query(User).all() # 1 query
for user in users:
print(user.posts) # N queries (1 per user)
# GOOD — eager loading with joinedload
from sqlalchemy.orm import joinedload
users = (
session.query(User)
.options(joinedload(User.posts))
.all()
) # 1 query
for user in users:
print(user.posts) # No additional queries
Database operations can fail for many reasons: deadlocks, timeouts, constraint violations, server restarts. Always wrap database calls in try/except and handle failures gracefully.
Never store raw passwords. Always hash them with a salt. Use bcrypt or argon2 in production — our example used SHA-256 for simplicity, but dedicated password hashing libraries are much more secure.
mysql.connector.Error, log the details, and fail gracefully. Do not let raw database errors leak to your users.os.environ or a secrets manager.%s placeholders) are mandatory — they prevent SQL injection and should be your default.commit() / rollback()) ensure data consistency for multi-statement operations.With these patterns and practices in your toolkit, you can confidently build Python applications backed by MySQL — from quick scripts to production web services.