Personal notes on Rust error guidelines, handling, and tips for library and application errors.

TL;DR

Core Principles:

• Use enum types with #[non_exhaustive] for error definitions
• Wrap third-party errors—never expose them directly in your public API
• Implement Display and Error traits (or use thiserror to reduce boilerplate)
• Create Result<T> type aliases for cleaner function signatures

Key Decisions:

• Libraries: Use thiserror for custom error types with minimal boilerplate
• Applications: Use anyhow for flexible error handling with context
• CLI Tools: Consider miette or eyre for rich, user-friendly error diagnostics
• Recoverable vs Unrecoverable: Use Result for recoverable errors, panic! for programming errors

Essential Patterns:

• Implement From trait for automatic error conversion with ? operator
• Add context to errors using map_err() or .context() (with anyhow)
• Document error conditions in function docs with # Errors section
• Use helper methods for common error construction patterns

Recommendations for Creating Error Types

1. Use an enum for Your Error Type

Define errors as enum to represent multiple distinct error conditions:

pub enum MyError {
    IoError(std::io::Error),
    ParseError(String),
    ValidationError { field: String, reason: String },
}

Why? Enums provide type-safe error handling and enable exhaustive pattern matching.

2. Group Related Error Conditions

Organize error variants logically within as few enums as necessary:

pub enum DatabaseError {
    ConnectionFailed(String),
    QueryFailed { query: String, cause: String },
    RecordNotFound { id: u64 },
    Timeout,
}

Why? This improves maintainability and keeps error types focused on a single domain.

3. Encapsulate Third-Party Errors

Don't expose error types from external dependencies directly:

// Bad: Exposes implementation details
pub fn read_config() -> Result<Config, std::io::Error> { }

// Good: Wraps external error
pub fn read_config() -> Result<Config, ConfigError> { }

pub enum ConfigError {
    Io(std::io::Error),
    Parse(serde_json::Error),
}

Why? This decouples your API from implementation details and allows error type evolution.

4. Make Your Enum Non-Exhaustive

Use #[non_exhaustive] to allow adding variants without breaking changes:

#[non_exhaustive]
pub enum ApiError {
    Unauthorized,
    NotFound,
    ServerError,
}

Why? This maintains backward compatibility when adding new error variants in future releases.

5. Implement Required Traits

5a. Manual Implementation

Will require implementing std::fmt::Display and std::error::Error:

use std::fmt;
use std::error::Error;

impl fmt::Display for MyError {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match self {
            MyError::IoError(e) => write!(f, "IO error: {}", e),
            MyError::ParseError(msg) => write!(f, "Parse error: {}", msg),
        }
    }
}

// Implementing source will enable error chaining
// https://doc.rust-lang.org/std/error/trait.Error.html#method.source
impl Error for MyError {
    fn source(&self) -> Option<&(dyn Error + 'static)> {
        match self {
            MyError::IoError(e) => Some(e),
            _ => None,
        }
    }
}

5b. Using thiserror (Recommended)

An alternative is to use the thiserror package to reduce boilerplate:

use thiserror::Error;

#[derive(Error, Debug)]
#[non_exhaustive]
pub enum MyError {
    #[error("IO error: {0}")]
    IoError(#[from] std::io::Error),

    #[error("Parse error: {0}")]
    ParseError(String),

    #[error("Validation failed for field '{field}': {reason}")]
    ValidationError { field: String, reason: String },
}

Why? The Error trait enables compatibility with ? operator and error handling libraries. thiserror reduces boilerplate significantly.

Additional Tips

Use Result Type Aliases

Create type aliases using Result for cleaner and more consistent function signatures:

pub type Result<T> = std::result::Result<T, UserServiceError>;

pub fn get_user(id: u64) -> Result<User> {
    // ...
}

Why? This reduces repetition and makes refactoring easier if you need to change the error type.

Implement From Trait for Error Conversion

The From trait helps with automatic error conversion:

// Manual implementation
impl From<std::io::Error> for MyError {
    fn from(error: std::io::Error) -> Self {
        MyError::Io(error)
    }
}

impl From<serde_json::Error> for MyError {
    fn from(error: serde_json::Error) -> Self {
        MyError::Parse(error.to_string())
    }
}

// Or use thiserror's #[from] attribute
#[derive(Error, Debug)]
pub enum MyError {
    #[error("IO error: {0}")]
    Io(#[from] std::io::Error),

    #[error("Parse error: {0}")]
    Parse(#[from] serde_json::Error),
}

// Now you can use ? operator with automatic conversion
fn read_and_parse() -> Result<Config, MyError> {
    let content = std::fs::read_to_string("config.json")?; // auto-converts io::Error
    let config = serde_json::from_str(&content)?; // auto-converts serde_json::Error
    Ok(config)
}

Why? Automatic conversion eliminates manual map_err() calls and makes error propagation seamless.

Add Context with Error Chaining

fn load_config(path: &str) -> Result<Config, ConfigError> {
    let content = std::fs::read_to_string(path)
        .map_err(|source| ConfigError::ReadFailed {
            path: path.to_string(),
            source,
        })?;

    let config = serde_json::from_str(&content)
        .map_err(|source| ConfigError::ParseFailed {
            path: path.to_string(),
            source,
        })?;

    Ok(config)
}

Why? Context-rich errors make debugging significantly easier by preserving the error chain.

Consider Error Codes for APIs

#[derive(Error, Debug)]
#[non_exhaustive]
pub enum ApiError {
    #[error("[AUTH001] Unauthorized: {0}")]
    Unauthorized(String),

    #[error("[RES404] Resource not found: {resource_type} with id {id}")]
    NotFound { resource_type: String, id: String },
}

impl ApiError {
    pub fn error_code(&self) -> &str {
        match self {
            ApiError::Unauthorized(_) => "AUTH001",
            ApiError::NotFound { .. } => "RES404",
        }
    }
}

Why? Error codes enable clients to handle errors programmatically and support internationalization.

Document Error Conditions

/// Fetches a user by their ID from the database.
///
/// # Errors
///
/// This function will return an error if:
/// - [`UserServiceError::Database`] - Database connection or query fails
/// - [`UserServiceError::UserNotFound`] - No user exists with the given ID
/// - [`UserServiceError::PermissionDenied`] - Caller lacks read permissions
pub fn get_user(id: u64) -> Result<User, UserServiceError> {
    // ...
}

Why? Documentation helps users understand and handle errors correctly without reading the source code.

Use #[must_use] for Result Types

When necessary use #[must_use] to enforce handling of results:

#[must_use = "this function returns a Result that should be handled"]
pub fn save_user(user: &User) -> Result<()> {
    // ...
}

Why? This prevents accidentally ignoring errors, which is a common source of bugs.

Create Helper Methods for Common Patterns

use thiserror::Error;

#[derive(Error, Debug)]
#[non_exhaustive]
pub enum ValidationError {
    #[error("Field '{field}' is required")]
    Required { field: String },

    #[error("Field '{field}' has invalid format: {reason}")]
    InvalidFormat { field: String, reason: String },
}

// Helper methods
impl ValidationError {
    pub fn required(field: impl Into<String>) -> Self {
        Self::Required { field: field.into() }
    }

    pub fn invalid_format(field: impl Into<String>, reason: impl Into<String>) -> Self {
        Self::InvalidFormat {
            field: field.into(),
            reason: reason.into(),
        }
    }
}

// Usage
fn validate_email(email: &str) -> Result<(), ValidationError> {
    if email.is_empty() {
        return Err(ValidationError::required("email"));
    }
    if !email.contains('@') {
        return Err(ValidationError::invalid_format("email", "missing @ symbol"));
    }
    Ok(())
}

Why? Helper methods reduce boilerplate and ensure consistent error construction.

Separate Recoverable from Unrecoverable Errors

Use Result for recoverable errors and panic! for unrecoverable ones:

// Recoverable: User might fix the input
pub fn parse_age(input: &str) -> Result<u32, ParseError> {
    input.parse().map_err(|_| ParseError::InvalidAge(input.to_string()))
}

// Unrecoverable: Programming error
pub fn get_config_value(key: &str) -> String {
    CONFIG.get(key)
        .unwrap_or_else(|| panic!("Config key '{}' must be set", key))
}

Why? This distinction clarifies error handling strategies and improves code readability.

Additional Libraries and Tools

Use anyhow for Application-Level Error Handling

For applications (not libraries), consider using anyhow:

use anyhow::{Context, Result};

fn load_config(path: &str) -> Result<Config> {
    let content = std::fs::read_to_string(path)
        .context(format!("Failed to read config from '{}'", path))?;

    let config: Config = serde_json::from_str(&content)
        .context(format!("Failed to parse config from '{}'", path))?;

    Ok(config)
}

Why? anyhow simplifies error handling by providing a generic error type that can wrap any error, along with context.

Enhanced Error Reporting with miette

For applications that need rich diagnostic error reporting try miette:

use miette::{Diagnostic, Result};
use thiserror::Error;

#[derive(Error, Debug, Diagnostic)]
#[error("Configuration validation failed")]
#[diagnostic(
    code(config::validation_failed),
    help("Check your configuration file syntax and required fields")
)]
pub struct ConfigValidationError {
    #[source_code]
    src: String,

    #[label("This field is invalid")]
    invalid_field: miette::SourceSpan,
}

fn validate_config(config_content: &str) -> Result<Config> {
    // ... validation logic
    Err(ConfigValidationError {
        src: config_content.to_string(),
        invalid_field: (42, 10).into(), // byte offset and length
    })?
}

Why? miette provides beautiful, IDE-like error reports with source code snippets, suggestions, and structured diagnostic information. Useful for CLI tools and developer-facing applications.

Error Tracing Integration

Integrate errors with structured logging using tracing-error:

use tracing_error::ErrorLayer;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

fn main() -> Result<()> {
    tracing_subscriber::registry()
        .with(ErrorLayer::default())
        .with(tracing_subscriber::fmt::layer())
        .init();

    // Your application code
    Ok(())
}

// In your error handling code
match operation() {
    Ok(result) => result,
    Err(e) => {
        tracing::error!(
            error = ?e,
            operation = "daily_backup",
            "Operation failed"
        );
        return Err(e.into());
    }
}

Why? This allows capturing rich context about errors in logs, facilitating debugging and monitoring in production environments.

DFS channels are part of the 5GHz band and are shared with radar systems. When a radar signal is detected, the router must switch to a non-DFS channel to avoid interference. This can lead to a decrease in network performance, especially if the new channel is crowded. ^{1 2}

From personal experience, I've noticed that my Asus router sometimes switches to a non-DFS channel, but very rarely, since I don't live near an airport or a military base and I'm not entirely sure what triggers the switch.

Setting Up Automatic Channel Checking

To automate the process of checking and restarting the WiFi channel, you can use a script that periodically checks if the channel has changed and restarts the WiFi if necessary. Here's how you can set it up:

Step 1: Create the channel-ck.sh Script

First, SSH into your router and navigate to the /jffs/scripts/ directory. Create a new script file named channel-ck.sh:

asus@RT-AX88U_Pro:/tmp/home/root# cd /jffs/scripts/
asus@RT-AX88U_Pro:/jffs/scripts# vi channel-ck.sh

Add the following content to the script:

#!/bin/sh

logger -t "channel_check" "Checking..."

ifname="$(nvram get wl1_ifname)"
targchanspec="$(nvram get wl1_chanspec)"
targchan=${targchanspec%/*}
currchanspec="$(wl -i $ifname chanspec | awk '{print $1}')"
currchan=${currchanspec%/*}

if [ "$targchan" != "0" ] && [ "$currchan" != "$targchan" ]; then
    logger -t "channel_check" "Channel has changed from ${targchan} to ${currchan}. Restarting WiFi."
    service restart_wireless
fi

logger -t "channel_check" "Done."

Make the script executable:

chmod a+rx /jffs/scripts/channel-ck.sh

Step 2: Schedule the Script with cru

You can use the cru command to schedule the script to run at specific intervals. This can be done by updating the services-start file or directly using the cru command.

Option 1: Update services-start File

Edit the services-start file in the /jffs/scripts/ directory:

asus@RT-AX88U_Pro:/jffs/scripts# vi services-start

Add the following lines to schedule the script:

#!/bin/sh

# Workdays (Monday to Friday) at 20 minutes past every hour from 0-8 AM and 7-11 PM
/usr/sbin/cru a channel_ck_weekdays "20 0-8,19-23 * * 1-5 sh /jffs/scripts/channel-ck.sh"

# Weekends (Saturday and Sunday) at 20 minutes past every hour
/usr/sbin/cru a channel_ck_weekends "20 * * * 6,0 sh /jffs/scripts/channel-ck.sh"

Option 2: Use cru Command Directly

Alternatively, you can use the cru command directly in the terminal to set up the cron jobs:

/usr/sbin/cru a channel_ck_weekdays "20 0-8,19-23 * * 1-5 sh /jffs/scripts/channel-ck.sh"
/usr/sbin/cru a channel_ck_weekends "20 * * * 6,0 sh /jffs/scripts/channel-ck.sh"

End

By setting up this script and scheduling it with cru, you can ensure that your router automatically checks and restarts the WiFi 5GHz if it switches due to DFS. ^{3 4}

See the original posts that inspired me to combine the found information:

• https://www.reddit.com/r/linux_gaming/comments/199ylqz/streaming_with_sunshine_from_virtual_screens/
• https://www.reddit.com/r/linuxmasterrace/comments/eraedc/after_an_evening_of_furious_googling_i_figured/
• https://www.reddit.com/r/pop_os/comments/soo7eh/how_to_permanently_load_custom_edid_from_boot_to/

Why Use a Virtual Display?

• Cost-effective: No need to buy dummy plugs.
• Customizable resolutions: Stream in resolutions your physical monitor may not support.
• Convenience: Keep your monitor off while streaming.

Step 1: Get an EDID File

An EDID (Extended Display Identification Data) file simulates a monitor. You can download pre-made EDID files from the v4l-utils repository. Choose one that matches your desired resolution and features (e.g., 4K, HDR).

Step 2: Place the EDID File

1. Create a directory for EDID files:

   sudo mkdir -p /usr/lib/firmware/edid
   

2. Copy your chosen EDID file to this directory. For example:

   sudo cp samsung-q800t-hdmi2.1 /usr/lib/firmware/edid/
   

Step 3: Configure Kernel Parameters

1. Identify your GPU's free HDMI or DP output:

   for p in /sys/class/drm/*/status; do con=${p%/status}; echo -n "${con#*/card?-}: "; cat $p; done
   

   Replace HDMI-A-1 in the next step with the appropriate output.

2. Add the following kernel parameters (see arch wiki or endeavour os discovery):

   drm.edid_firmware=HDMI-A-1:edid/samsung-q800t-hdmi2.1 video=HDMI-A-1:e
   

3. Update your mkinitcpio.conf file to include the EDID file:

   FILES=(/usr/lib/firmware/edid/samsung-q800t-hdmi2.1)
   

4. Regenerate the initramfs:

   sudo mkinitcpio -P
   

5. Reboot your system.

Step 4: Use Sunshine for Streaming

After rebooting, Sunshine can use the virtual display for streaming. Ensure you configure Sunshine to capture the virtual display using KMS or wlroots capture.

Step 5: Automate with Custom Scripts

You can enhance your setup by adding scripts to Sunshine. These scripts can improve your experience with sunshine by automating the resolution set up and toggling off and on of the main monitor display. Examples are available in this GitHub Gist.

Example Script Workflow:

• On Connect: Disable physical monitors and enable the virtual display.
• On Disconnect: Re-enable physical monitors and disable the virtual display.

Conclusion

By following these steps, you can create a virtual display for Sunshine on Arch Linux without spending extra money on dummy plugs. This setup is flexible, allowing you to stream in higher resolutions or HDR, and can be further automated with custom scripts for a seamless experience. For bonus points you can install and use wxedid to modify the edid file and adjust the display parameters such as hz or resolutions, but that is outside the scope of this article.

My Setup

The Problem

Despite my monitor advertising VRR support, the option to enable it in KDE was missing. I tried several things, including:

• Verifying VRR support in the monitor's EDID file.

  # 1. Install the read-edid package
  sudo pacman -S read-edid
  # 2. Use get-edid and parse-edid to read and decode the monitor's EDID
  sudo get-edid | parse-edid
  

• Updating drivers and ensuring my system was fully up-to-date.

• Tweaking various settings in KDE and the AMD GPU configuration. And even trying the CableMatters DisplayPort to HDMI Adapter to no avail.

The EDID file confirmed that my monitor does support VRR (VertRefresh 48-120), but nothing I tried worked. And the kscreen-doctor --outputs command was showing Vrr: incapable output.

The Solution: Disabling MST

After much trial and error, I discovered the culprit: MST (Multi-Stream Transport). My monitor had MST enabled by default, which is typically used for daisy-chaining multiple displays, which I wasn't using. Disabling MST in the monitor’s settings immediately resolved the issue, and the VRR option became available in KDE.

Steps to Disable MST

1. Open your monitor’s on-screen display (OSD) menu.
2. Navigate to the settings related to MST or DisplayPort configuration.
3. Disable MST (it might be labeled as "Dualport" or "Multi-Stream Transport").
4. Save the changes and restart your system (just to be sure).

Possible Cause

This issue might be related to a bug in the AMD Radeon drivers or how MST interacts with VRR on Linux. MST can sometimes interfere with certain features, and it seems VRR is one of them in this case.

Final Thoughts

If you’re struggling to enable VRR on Linux and have a similar setup, check your monitor’s MST setting. Disabling it worked for me, and I hope it helps you too. While the root cause might be driver-related, I’m just happy to have VRR working now.

If you’ve encountered similar issues or have additional insights, feel free to share them with me!

The CKAD is a certificate from the Cloud Native Computing Foundation (CNCF). It checks if you can design, build, and manage applications for Kubernetes, which is a tool for running apps in a cloud environment. What's cool about this exam is it's all about doing, not just answering questions. You have to show you can actually work with Kubernetes.

The Exam

• Time: The exam lasts 2 hours.
• You'll work on tasks like:
  • Building container images
  • Deploying apps
  • Keeping an eye on apps and fixing them if needed
  • Setting up the environment
  • Managing services and networks

I really liked that the exam is practical. Instead of choosing from answers, you work on tasks like you would at work. It's challenging because you're under time pressure.

How I Prepared

Here are some things that helped me get ready:

• GitHub: There are lots of free study materials and practice tasks from other developers on GitHub.
  • https://github.com/dgkanatsios/CKAD-exercises
  • https://github.com/bmuschko/ckad-crash-course
• Courses: I took some courses on Udemy that went through everything I needed to know. They had practice exercises too.
  • https://www.udemy.com/course/certified-kubernetes-application-developer/
  • https://www.udemy.com/course/kubernetes-microservices/
• Practice: I set up Kubernetes on my computer using minikube and practiced. You need to be comfortable with kubectl and learn tricks on how to speed up your workflow to be able to do the exam in 2 hours.
  • https://killer.sh is a good resource for a practice exam, but it is much more difficult than the real one. Good way to see how well you're doing under time pressure.

I might have prepared more than I needed to, but it was worth it. I learned a lot about Kubernetes, which will definitely help me in the future.

What’s Next?

I'd like to get CKA: Certified Kubernetes Administrator and CKS: Certified Kubernetes Security Specialist next. We'll see how it goes.

If you have any further questions, feel free to reach out to me.

Thank you for reading.

Steps for updating the Proton VPN WireGuard configuration

1. Download your Proton VPN WireGuard config from the Proton VPN WireGuard configuration.
2. Get your NextDNS configuration ID and Endpoints from the NextDNS dashboard.
3. In the downloaded wireguard config remove the DNS = line from the [Interface] section.
4. Add a PostUp command to configure NextDNS using resolvectl and your NextDNS ID and Endpoints: PostUp = resolvectl dns %i 172.16.80.142#ubuntu-1234a5.dns.nextdns.io 2001:db8:a0b:12f0::1#ubuntu-1234a5.dns.nextdns.io 172.16.83.142#ubuntu-1234a5.dns.nextdns.io 2001:db8:a0b:12f1::1#ubuntu-1234a5.dns.nextdns.io; resolvectl dnsovertls %i yes; resolvectl domain %i ~.
5. Save the modified config file.

Here's how it might look:

[Interface]
# Key for GB#1
# Bouncing = 0
# NetShield = 0
# Moderate NAT = off
# NAT-PMP (Port Forwarding) = off
# VPN Accelerator = on
PrivateKey = <PrivateKey>
Address = <Address>
PostUp = resolvectl dns %i 172.16.80.142#ubuntu-1234a5.dns.nextdns.io 2001:db8:a0b:12f0::1#ubuntu-1234a5.dns.nextdns.io 172.16.83.142#ubuntu-1234a5.dns.nextdns.io 2001:db8:a0b:12f1::1#ubuntu-1234a5.dns.nextdns.io; resolvectl dnsovertls %i yes; resolvectl domain %i ~.

[Peer]
# GB#1
PublicKey = <PublicKey>
AllowedIPs = 0.0.0.0/0
Endpoint = <Address>

NB: I removed personal values and used random ones for IPv4, IPv6 and NextDNS ID. Make sure to use your own.

This setup leverages systemd-resolved, which some Linux distributions use by default. The PostUp command configures NextDNS when the WireGuard connection is established.

Now you can enjoy the combined benefits of Proton VPN's security and NextDNS's advanced filtering and analytics!

Managing WireGuard Configuration and Connection

Configuration File Location

After modifying your WireGuard configuration file, you should place it in the appropriate directory:

1. Save the configuration file with a .conf extension (e.g., wg0.conf).
2. Move the file to /etc/wireguard/. You may need root privileges for this:

   sudo mv wg0.conf /etc/wireguard/
   

WireGuard CLI Commands

Here are some useful WireGuard CLI commands to manage your VPN connection:

1. Start the WireGuard connection:

   sudo wg-quick up wg0
   

2. Stop the WireGuard connection:

   sudo wg-quick down wg0
   

3. Check the status of your WireGuard interfaces:

   sudo wg show
   

Enabling/Disabling Autostart on System Boot

To enable autostart of your WireGuard connection on system boot:

1. Enable the WireGuard service:

   sudo systemctl enable wg-quick@wg0.service
   

2. Start the service immediately:

   sudo systemctl start wg-quick@wg0.service
   

To disable autostart:

1. Disable the WireGuard service:

   sudo systemctl disable wg-quick@wg0.service
   

2. Stop the service if it's currently running:

   sudo systemctl stop wg-quick@wg0.service
   

Remember to replace wg0 in these commands with the actual name of your configuration file (without the .conf extension) if it's different.

And that's about it. You're all set up to use Proton VPN WireGuard with NextDNS on Linux.

You can use https://ip.me to verify that your IP address is now that of the Proton VPN server you're connected to. And https://test.nextdns.io or https://ping.nextdns.io to verify that your DNS queries are now going through NextDNS.

If you have any further questions or need help with this setup, feel free to send me an email. I'm always happy to help!

I found several solutions which implemented a varying width pagination, but I didn't like that the pagination panel was changing its size during use.

So, here is an implementation of a fixed width pagination in a Spring Boot project that uses Thymeleaf.

Pagination with an ellipsis	Pagination with an ellipsis on both sides

Project

You can find the project on https://github.com/azdanov/spring-pagination.

These parts are covered in the current article:

src/main/java/dev/azdanov/pagination/user
├── UserController.java
├── UserService.java
├── UserRepository.java
src/main/resources/templates
├── users.html
├── fragments
│   └── pagination.html
src/main/java/dev/azdanov/pagination/util
├── PaginationUtils.java
├── PageItem.java

UserController

The UserController handles HTTP requests and interacts with the UserService to fetch paginated data. It uses Pageable to pass pagination information, such as page size and current page number, to the service for an offset based pagination. A PagedModel is returned to the view, because Page is not stable and can change its implementation between Spring versions.

package dev.azdanov.pagination.user;

import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Pageable;
import org.springframework.data.web.PagedModel;
import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;

@Controller
@RequestMapping("/users")
@RequiredArgsConstructor
public class UserController {

    private final UserService userService;

    @GetMapping
    public String listUsers(Pageable pageable, Model model) {
        PagedModel<UserDto> users = userService.getUsers(pageable);

        model.addAttribute("users", users);
        model.addAttribute("pageable", pageable);

        return "users";
    }
}

UserService

The UserService fetches paginated user data from the repository and maps it to DTOs.

package dev.azdanov.pagination.user;

import lombok.RequiredArgsConstructor;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
import org.springframework.data.web.PagedModel;
import org.springframework.stereotype.Service;

@Service
@RequiredArgsConstructor
public class UserService {

    private final UserRepository userRepository;

    public PagedModel<UserDto> getUsers(Pageable pageable) {
        Page<UserDto> users = userRepository.findAll(pageable)
            .map(user -> new UserDto(user.getId(), user.getName(), user.getEmail()));

        return new PagedModel<>(users);
    }
}

UserRepository

The UserRepository extends JpaRepository to provide CRUD operations and pagination support. Default methods like findAll(Pageable pageable) are used to fetch paginated data, so it's quite easy to implement pagination in Spring Data JPA.

package dev.azdanov.pagination.user;

import org.springframework.data.jpa.repository.JpaRepository;
import org.springframework.stereotype.Repository;

@Repository
public interface UserRepository extends JpaRepository<User, Long> {
}

Thymeleaf Template

The users.html template displays the list of users and includes the pagination controls.

As a side note, I use the Thymeleaf Layout Dialect to create a layout structure with a main layout and separate pages. The layout:decorate attribute is used to apply the layout to the current template.

And Bootstrap 5.3 is used for styling.

<!DOCTYPE html>
<html
  xmlns:th="http://www.thymeleaf.org"
  xmlns:layout="http://www.ultraq.net.nz/thymeleaf/layout"
  layout:decorate="~{layouts/main}"
  lang="en"
>
  <head>
    <title>Users</title>
  </head>
  <body>
    <div layout:fragment="content">
      <h1 class="mb-4">Users</h1>
      <p>
        This table's pagination uses Spring Data JPA Pageable offset based
        pagination:
        <a
          href="https://docs.spring.io/spring-data/jpa/reference/repositories/query-methods-details.html#repositories.special-parameters"
          >Spring Data JPA - Paging, Iterating Large Results, Sorting &
          Limiting</a
        >
      </p>
      <div class="table-responsive">
        <table class="table-striped table-hover table">
          <thead class="table-dark">
            <tr>
              <th>ID</th>
              <th>Name</th>
              <th>Email</th>
            </tr>
          </thead>
          <tbody>
            <tr th:each="user : ${users.content}">
              <td th:text="${user.id}"></td>
              <td th:text="${user.name}"></td>
              <td th:text="${user.email}"></td>
            </tr>
          </tbody>
        </table>
      </div>

      <div
        th:replace="~{fragments/pagination :: offset-pagination(${users}, ${pageable})}"
      ></div>
    </div>
  </body>
</html>

Pagination Fragment

The pagination.html fragment handles the pagination controls and navigation. There are a lot of stuff going on, so let's break it down:

• The pagedModel and pageable objects are passed to the fragment.
• Various helper variables are created to simplify the code with th:with.
• An info text is displayed with the current page range and total elements.
• It generates the pagination structure where the PaginationUtils class is used to calculate the page items.
• It includes a select element to change the page size and some JavaScript to handle the page size change event (currentUrl is globally set).

<nav
  th:fragment="offset-pagination(pagedModel, pageable)"
  th:with="page=${pagedModel.getMetadata()}"
  id="pagination"
  aria-label="Page navigation"
  class="mt-2 mb-5"
>
  <div class="d-flex justify-content-between align-items-center">
    <div
      class="text-muted d-none d-sm-block text-nowrap"
      th:with="start=${page.number * page.size + 1}, end=${T(java.lang.Math).min((page.number + 1) * page.size, page.totalElements)}"
    >
      <span th:text="|${start}-${end} of ${page.totalElements}|"></span>
    </div>

    <ul
      class="pagination mb-0"
      th:if="${!pagedModel.getContent().isEmpty() && page.totalPages > 1}"
      th:with="currentPage = ${page.number}, totalPages = ${page.totalPages}"
    >
      <li class="page-item" th:classappend="${currentPage == 0} ? 'disabled'">
        <a
          class="page-link"
          th:tabindex="${currentPage == 0} ? '-1' : '0'"
          th:href="@{${currentUrl}(page=${currentPage - 1}, size=${pageable.pageSize})}"
          aria-label="Previous"
        >
          <span aria-hidden="true">&laquo;</span>
        </a>
      </li>

      <li
        class="page-item"
        th:each="pageItem : ${T(dev.azdanov.pagination.util.PaginationUtils).getPageItems(page)}"
        th:classappend="${pageItem.pageNumber == currentPage + 1} ? 'active'"
      >
        <a
          class="page-link"
          th:href="@{${currentUrl}(page=${pageItem.pageNumber - 1}, size=${pageable.pageSize})}"
          th:text="${pageItem.isEllipsis() ? '&hellip;' : pageItem.pageNumber}"
          th:aria-label="${'Page ' + pageItem.pageNumber}"
        ></a>
      </li>

      <li
        class="page-item"
        th:classappend="${currentPage == totalPages - 1} ? 'disabled'"
      >
        <a
          class="page-link"
          th:tabindex="${currentPage == totalPages - 1} ? '-1' : '0'"
          th:href="@{${currentUrl}(page=${currentPage + 1}, size=${pageable.pageSize})}"
          aria-label="Next"
        >
          <span aria-hidden="true">&raquo;</span>
        </a>
      </li>
    </ul>

    <div class="d-flex align-items-center">
      <label
        class="text-muted d-none d-sm-block pe-2 text-nowrap"
        for="paginationSize"
      >
        Page size
      </label>
      <select
        class="form-select form-select w-auto text-center"
        name="paginationSize"
        id="paginationSize"
      >
        <option
          th:each="size : ${ { 5, 10, 20, 50} }"
          th:value="${size}"
          th:text="${size}"
          th:selected="${page.size == size}"
        ></option>
      </select>
    </div>
  </div>

  <script th:inline="javascript">
    document.addEventListener("DOMContentLoaded", function () {
      const paginationSizeSelect = document.getElementById("paginationSize");
      const currentUrl = /*[[${currentUrl}]]*/ "";

      paginationSizeSelect.addEventListener("change", function () {
        const newSize = this.value;
        const url = new URL(window.location.origin + currentUrl);

        url.searchParams.set("page", "0");
        url.searchParams.set("size", newSize);

        window.location.href = url.toString();
      });
    });
  </script>
</nav>

PaginationUtils

The PaginationUtils class generates the pagination structure, including ellipses for large datasets.

I found an algorithm online: https://gist.github.com/kottenator/9d936eb3e4e3c3e02598?permalink_comment_id=5146730#gistcomment-5146730

Credits to the author: https://gist.github.com/pk936. I modified it to fit my needs.

Although it had a small bug, where for smaller page sizes it showed the ellipsis on the right side, even though it could display all the pages. I added some additional checks to fix the issue.

I liked this implementation because it has fixed width and doesn't expand or shrink as you navigate through the pages, compared to other solutions I found.

package dev.azdanov.pagination.util;

import org.springframework.data.web.PagedModel;
import org.springframework.lang.NonNull;

import java.util.ArrayList;
import java.util.List;
import java.util.stream.IntStream;

public final class PaginationUtils {

    private static final int CENTER_PAGES = 3;
    private static final int ELLIPSIS_THRESHOLD = 2;

    private PaginationUtils() {
    }

    public static List<PageItem> getPageItems(@NonNull PagedModel.PageMetadata metadata) {
        int currentPage = (int) (metadata.number() + 1);
        int totalPages = (int) metadata.totalPages();

        return calculatePagination(currentPage, totalPages);
    }

    private static List<PageItem> calculatePagination(int currentPage, int totalPages) {
        if (totalPages <= CENTER_PAGES + ELLIPSIS_THRESHOLD * 2) {
            return createFullPagination(totalPages);
        }

        int startPage = Math.max(currentPage - 1, 1);
        int endPage = Math.min(currentPage + 1, totalPages);

        boolean showLeftEllipsis = startPage > ELLIPSIS_THRESHOLD;
        boolean showRightEllipsis = endPage < totalPages - ELLIPSIS_THRESHOLD;

        if (!showLeftEllipsis && showRightEllipsis) {
            return createLeftPagination(totalPages);
        }
        if (showLeftEllipsis && !showRightEllipsis) {
            return createRightPagination(totalPages);
        }
        if (showLeftEllipsis && showRightEllipsis) {
            return createCenterPagination(startPage, endPage, totalPages);
        }

        return createFullPagination(totalPages);
    }

    private static List<PageItem> createLeftPagination(int lastPage) {
        int end = CENTER_PAGES + ELLIPSIS_THRESHOLD;

        List<PageItem> pages = new ArrayList<>(range(1, end));
        pages.add(new PageItem(end + 1, true));
        pages.add(new PageItem(lastPage, false));

        return pages;
    }

    private static List<PageItem> createRightPagination(int lastPage) {
        int start = lastPage - CENTER_PAGES - 1;

        List<PageItem> pages = new ArrayList<>();
        pages.add(new PageItem(1, false));
        pages.add(new PageItem(start - 1, true));
        pages.addAll(range(start, lastPage));

        return pages;
    }

    private static List<PageItem> createCenterPagination(int startPage, int endPage, int lastPage) {
        List<PageItem> pages = new ArrayList<>();
        pages.add(new PageItem(1, false));
        pages.add(new PageItem(startPage - 1, true));
        pages.addAll(range(startPage, endPage));
        pages.add(new PageItem(endPage + 1, true));
        pages.add(new PageItem(lastPage, false));

        return pages;
    }

    private static List<PageItem> createFullPagination(int lastPage) {
        return range(1, lastPage);
    }

    private static List<PageItem> range(int start, int end) {
        return IntStream.rangeClosed(start, end)
            .mapToObj(i -> new PageItem(i, false))
            .toList();
    }
}

PageItem

The PageItem class represents a single page in the pagination component. It helps us to distinguish between regular pages and ellipses. Another way is to return -1 instead of creating a new class, but then in the Thymeleaf fragment it is impossible to distinguish between a regular page and an ellipsis to make it a navigable link.

package dev.azdanov.pagination.util;

public record PageItem(int pageNumber, boolean isEllipsis) {}

Conclusion

I hope this article helps you with implementing a pagination in your projects. If you have any questions or suggestions, feel free to contact me.

Thank you for reading!

In the repository stratospheric-dev/hello-stratospheric, you can check all the parts of the pipeline.

The repository consists of three major areas:

• GitHub Actions for workflows.
• AWS CDK to manage AWS resources.
• Spring Boot (Docker) for publishing a new image.

GitHub Actions pipeline overview

For the pipeline, GitHub Actions can be used. There are many ways it can be organized, but in general, we want to have an automated deployment into a staging environment when a branch is merged. For changing the repository or network stack, it is fine to trigger them manually via a workflow.

Here's what it might look like. (Examples are from the Stratospheric demo repository)

Manual Steps:

1. Bootstrap AWS CDK.
2. Create the AWS CDK repository stack (ECR) for a specific app.
3. Created the AWS CDK network stack (VPC, IGW, ELB, ECS).

01-bootstrap.yml
02-create-environment.yml

Automated Steps (on branch merge):

1. Test and build the Spring Boot app.
2. Create and push a Docker image into the AWS CDK repository stack.
3. Deploy the AWS CDK service stack.

03-build-and-deploy.yml

As we can see, the node.js npm is executing the AWS CDK, making the GitHub Actions configuration files quite easy to read.

AWS CDK overview

The AWS CDK project can be found in main/cdk. It uses Node.js package.json file to expose available deployments, and cdk.json to provide parameters.

One interesting feature of ASW CDK is the ability to abstract and encapsulate deployment code into a construct library, which can be reused in different projects and teams.

The stratospheric-dev/cdk-constructs library will be used to import the DockerRepository, Network, ApplicationEnvironment, and Service constructs inside Java files which will create a single or multiple grouped AWS resources.

Spring Boot overview

Two things need to happen for the Spring Boot app.

First, the test and build steps need to produce a jar file, and then the docker image must be created, tagged, and pushed into ECR.

1. Test and build the app.
2. Create the docker image. Dockerfile
3. Tag the images and push them to ECR.

Conclusion

This is a rough outline of the steps needed for an automated AWS CDK pipeline. The hardest part is the AWS CDK, where knowledge of individual resources, settings, and how they integrate together is important for a successful deployment.

As a side note, Cantrill - AWS Certified Solutions Architect - Associate (SAA-C03) is an excellent resource to get started with AWS, either to prepare for the exam or to gain general AWS knowledge.

Stratospheric: From Zero to Production with Spring Boot and AWS

I've found a book called Stratospheric about learning AWS CDK for a Spring Boot app. It was in my backlog, but I finally started going through it.

This book provides a step-by-step guide on how to build a Spring Boot app and deploy it to AWS. It explains the concepts of CloudFormation, and then it shows how to use AWS CDK to create the infrastructure for the app.

I've yet to finish the book, but I've already learned more about using AWS CDK and AWS in general. I had limited experience with CloudFormation, which I didn't know how to use correctly, and no experience with AWS CDK. Previously, I only used the Serverless Framework and Terraform to deploy my apps to AWS.

Next Steps

As I continue reading the book, I'll write more about my experience with AWS CDK, AWS, and Spring Boot.

I think it was a good idea to pass AZ-104 first, because it gave me a good understanding of the Azure ecosystem. I would recommend doing the same. Also, there seems to be a lack of good preparation material for AZ-204. The official Microsoft Learn barely covers the exam topics, considering how tricky the questions were.

I liked the topics covered in the exam, because they were not part of the AZ-104. The queue, event grid and service bus topics were very interesting to learn about. Security is still and identity management is still quite tricky for me, but I am getting there. And I'd argue that Cosmos DB is the most difficult topic of all. Cosmos DB can be very easy to start with, but there are many features and not so much overall experience or material to learn from. So learning relational SQL is much easier and worthwhile.

Preparation

I have used the following resources:

• Udemy - AZ-204: Developing Solutions for Microsoft Azure - A course from Alan Rodrigues. It was okay, but very tedious and code examples were not so good. But it covers a lot of topics if you can stomach sitting through 27 hours of video.
• Udemy - Microsoft Azure for .NET Developers - A course by Trevoir Williams. It was exceptional and engaging. Sadly it only covers a small portion of the exam topics. Yet I highly recommend it, because it covers how to use some of the services in .NET in a very practical way.
• Microsoft Learn: Exam AZ-204: Microsoft Azure Developer Associate - Official training material, with free a sandbox to practice hands-on.
• MeasureUp: Microsoft Practice Test AZ-204: Microsoft Azure Developer - Some additional practice exams. They were better than AZ-104, but I'd recommend to find additional practice tests on other resources.

Final thoughts

Take much more time than you think you would need for AZ-204. Preferably, if you can, take AZ-104 beforehand. I recommend doing cloudlee AZ-104 and John Savill AZ-104 courses. This should give you general picture about Azure and prepare you for AZ-104/204. After that, just reading documentation and following the exam guidelines would be the best way to prepare, apart from the resources I mentioned above (udemy, microsoft learn, measureup).

It took me a few weeks to finally write this blog post because this exam kicked my butt, and I had to take some time off. I am glad I passed it, but I was so certain that I failed during exam that it was quite a rollercoaster of emotions.

The Next one for me will be AZ-400, but I will take a break for a while. Change gears and study some C# and ASP.NET Core.

There were a bunch of questions about older Azure services, which I have never used, so I had to guess the answers, e.g., general-purpose v1 / blob storage (those will be deprecated soon). Or very specific questions which I couldn't find afterward even though I tried to find an answer in the documentation (virtual machine scale sets update domain related). And some questions were very basic, just selecting an appropriate service to use out of a list of options that should've been only in AZ-900.

I did enjoy the questions that required me to think more based on an example picture and figure out what's wrong with it, or to choose an appropriate solution. Or reading an ARM template or a Policy definition. I liked questions that you can logically think through and find the answer, instead of just memorizing some fact from documentation.

I would say that it was beneficial to pass AZ-900 first, to get a general understanding of Azure services, and to see how the exam itself is done. But I don't see myself doing any other 900-level exams in the future, only if I have a free voucher.

Preparation

I have used the following resources:

• Cloudlee: AZ-104 Microsoft Azure Administrator - Course is 23 hours long, and it covers all the topics in the exam. I went through this course 2 times, so good it was. I highly recommend it.
• Microsoft Learn: Exam AZ-104: Microsoft Azure Administrator Associate - Official training material, with free a sandbox to practice hands-on.
• YouTube: John Savill's AZ-104 Azure Administrator Study List - Another really amazing playlist on YouTube by John Savill.
• Tutorials Dojo: AZ-104 Microsoft Azure Administrator Practice Exams - Very useful practice exams by Tutorials Dojo.
• MeasureUp: Microsoft Practice Test AZ-104: Microsoft Azure Administrator - Some additional practice exams. I wouldn't recommend, too expensive and lots of irrelevant and difficult questions.

The next step for me is to get the Microsoft Certified: Azure Developer Associate (AZ-204) certificate.

Final thoughts

Why do I want to get certified? I don't think that it's a good way to prove your knowledge (it's too easy to find exam dumps and memorize the answers without understanding why), but it's a good way to structure your learning process. It forces you to learn things that you wouldn't learn otherwise (how would I find out that MARS exists on my own 😄?), and it gives you a deeper dive of the services that Azure provides.

I remember, when I was using AWS at my previous job, I was exposed and limited to only a few services, and I didn't know what else AWS could offer and didn't have time to learn more about the services that were being used or if there was a better alternative or some architectural pattern. Only recently I've learned about AWS Well-Architected Framework, and I see great value in learning how to use cloud services properly. Azure has a similar Microsoft Cloud Adoption Framework for Azure and Azure Well-Architected Framework which I would like to go over one day.

In the end, I'd like to find a job that uses cloud technologies extensively, and hopefully using Azure. I'm not sure how valuable the certificate itself will be, but studying and following the exam structure is helping me to get the experience needed for the job.

When logging in and creating a new organization, I navigated to Organization Settings > Parallel Jobs and saw that for Microsoft-hosted agents I have zero parallel jobs and for self-hosted agents I have one parallel job. Which meant that you can create your own VM and run your own agent on it, but for that you'll need to pay for the VM.

To get the free Microsoft-hosted agent, you'll need to fill out a form and wait for the approval: https://aka.ms/azpipelines-parallelism-request

I had a billing setup for my Azure account, which I also used for Azure DevOps, but maybe it wasn't necessary.

And you can read more about the configuration of parallel jobs here, and current request process for free grant: https://aka.ms/microsofthosted

I have spent around 10 days to prepare for it, with a final score of 925/1000. I have several years of experience with software development, and I have used AWS in production, so I was familiar with the concepts of cloud computing and cloud services.

Preparation

I have used the following resources:

• Microsoft Learn: Exam AZ-900: Microsoft Azure Fundamentals - Official training material, with free a sandbox to practice hands-on.
• YouTube: John Savill's AZ-900 Azure Fundamentals Certification Course - A really amazing course on YouTube by John Savill.
• Tutorials Dojo: AZ-900 Microsoft Azure Fundamentals Practice Exams - Very useful practice exams by Tutorials Dojo.

At the same time, I've found an interesting YouTube channel that focuses on Azure, DevOps, Terraform, and other related topics — Azure Terraformer by Mark Tinderholt. His videos are very useful and informative, for those interested in learning how to use Terraform with Azure ecosystem, and I highly recommend checking them out. Although it is about more advanced topics, it is still very helpful in starting to prepare for AZ-104 and AZ-400 exams.

The next step for me is to get the Microsoft Certified: Azure Administrator Associate (AZ-104) certificate.

Update: You can also watch a handy code review of the terraform project on YouTube by Azure Terraformer. Terraform Code Review: File Structure & Naming Conventions.

Prerequisites

AWS Account

You'll need an AWS account to host your website. If you don't have one, you can create one for free.

• Create an AWS account
• Getting started: Are you a first-time AWS user?
• Getting Started: Follow Security Best Practices as You Configure Your AWS Resources
• Avoiding unexpected charges
• What is the AWS Free Tier, and how do I use it?

Just to reiterate, you should follow the best practices when working with AWS:

1. Create a separate IAM user and don't use the root account.
2. Enable MFA for both the root account and the IAM user (or even better use SSO).
3. Create an AWS Budget to alert you when your costs exceed a certain threshold.

AWS CLI

You'll need to set up the AWS CLI and to configure your AWS credentials. If you don't have it installed, you can download it from the official website.

Terraform

You'll need Terraform to provision the infrastructure. If you don't have it installed, you can download it from the official website.

Domain Name

You'll need a domain name to host your website. If you don't have one, you can buy one from a domain registrar. In this article, I'll be using Route53 to manage my domain, because it will simplify the Terraform configuration.

There are ways to connect or migrate an existing domain name, but it's out of scope for this article.

We will be using a domain for the CloudFront distribution, and a TLS certificate.

Setup

After you've set up your AWS account, AWS CLI and Terraform, you can start provisioning the infrastructure.

I'll be using my Demo Next.js app as an example. You can find the source code on GitHub.

Setting up Terraform

Before we start, it's a good idea to manually create a DynamoDB table for Terraform state locking and an S3 bucket for Terraform remote state storage.

# Create an S3 bucket for Terraform remote state storage
aws s3api create-bucket --bucket azdanov-aws-static-website-demo-terraform-state --region us-east-1

# Disable public access to the S3 bucket
aws s3api put-public-access-block --bucket azdanov-aws-static-website-demo-terraform-state --public-access-block-configuration "BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true"

# Create a DynamoDB table for Terraform state locking
aws dynamodb create-table --table-name azdanov-aws-static-website-demo-terraform-lock-table --region us-east-1 --attribute-definitions AttributeName=LockID,AttributeType=S --key-schema AttributeName=LockID,KeyType=HASH --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

Writing Terraform file

Now, we can start writing the Terraform file. You can easily copy it from the GitHub repository.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.63"
    }
  }
  backend "s3" {
    bucket         = "azdanov-aws-static-website-demo-terraform-state"
    key            = "terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "azdanov-aws-static-website-demo-terraform-lock-table"
  }
}

provider "aws" {
  region = "us-east-1"
}

variable "domain_name" {
  type        = string
  description = "The domain name for the website, e.g. example.com. Must already be registered with Route53."
}

locals {
  origin_id = "${var.domain_name}-${sha1(var.domain_name)}"
}

# Route53 zone
data "aws_route53_zone" "zone" {
  name = var.domain_name
}


# S3 bucket
resource "aws_s3_bucket" "bucket" {
  bucket = substr(local.origin_id, 0, 63)

  tags = {
    Name = var.domain_name
  }
}

resource "aws_s3_bucket_public_access_block" "access" {
  bucket = aws_s3_bucket.bucket.id

  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}


# ACM certificate
resource "aws_acm_certificate" "certificate" {
  domain_name       = var.domain_name
  validation_method = "DNS"

  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_route53_record" "acm_record" {
  for_each = {
    for dvo in aws_acm_certificate.certificate.domain_validation_options : dvo.domain_name => {
      name   = dvo.resource_record_name
      record = dvo.resource_record_value
      type   = dvo.resource_record_type
    }
  }

  allow_overwrite = true
  name            = each.value.name
  records         = [each.value.record]
  ttl             = 60
  type            = each.value.type
  zone_id         = data.aws_route53_zone.zone.zone_id
}

resource "aws_acm_certificate_validation" "validation" {
  certificate_arn         = aws_acm_certificate.certificate.arn
  validation_record_fqdns = [for record in aws_route53_record.acm_record : record.fqdn]
}


# CloudFront distribution
resource "aws_cloudfront_origin_access_control" "oac" {
  name                              = "${var.domain_name} access control"
  description                       = "Cloudfront access control for the ${var.domain_name} distribution."
  origin_access_control_origin_type = "s3"
  signing_behavior                  = "always"
  signing_protocol                  = "sigv4"
}

data "aws_cloudfront_cache_policy" "cache_policy" {
  name = "Managed-CachingOptimized"
}

data "aws_cloudfront_origin_request_policy" "origin_request_policy" {
  name = "Managed-CORS-S3Origin"
}

data "aws_cloudfront_response_headers_policy" "response_headers_policy" {
  name = "Managed-SecurityHeadersPolicy"
}

resource "aws_cloudfront_distribution" "distribution" {
  enabled             = true
  is_ipv6_enabled     = true
  default_root_object = "index.html"
  http_version        = "http2and3"
  price_class         = "PriceClass_100"
  retain_on_delete    = false
  wait_for_deployment = false
  aliases             = [var.domain_name]

  origin {
    domain_name              = aws_s3_bucket.bucket.bucket_regional_domain_name
    origin_access_control_id = aws_cloudfront_origin_access_control.oac.id
    origin_id                = local.origin_id
  }

  custom_error_response {
    error_code         = "403"
    response_code      = "200"
    response_page_path = "/index.html"
  }

  default_cache_behavior {
    compress                   = true
    viewer_protocol_policy     = "redirect-to-https"
    allowed_methods            = ["GET", "HEAD"]
    cached_methods             = ["GET", "HEAD"]
    target_origin_id           = local.origin_id
    cache_policy_id            = data.aws_cloudfront_cache_policy.cache_policy.id
    origin_request_policy_id   = data.aws_cloudfront_origin_request_policy.origin_request_policy.id
    response_headers_policy_id = data.aws_cloudfront_response_headers_policy.response_headers_policy.id
  }

  restrictions {
    geo_restriction {
      restriction_type = "none"
    }
  }

  viewer_certificate {
    cloudfront_default_certificate = false
    acm_certificate_arn            = aws_acm_certificate.certificate.arn
    minimum_protocol_version       = "TLSv1.2_2021"
    ssl_support_method             = "sni-only"
  }
}

resource "aws_route53_record" "cloudfront_record" {
  zone_id = data.aws_route53_zone.zone.zone_id
  name    = var.domain_name
  type    = "A"

  alias {
    evaluate_target_health = false
    name                   = aws_cloudfront_distribution.distribution.domain_name
    zone_id                = aws_cloudfront_distribution.distribution.hosted_zone_id
  }
}

data "aws_iam_policy_document" "document" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${aws_s3_bucket.bucket.arn}/*"]

    principals {
      type        = "Service"
      identifiers = ["cloudfront.amazonaws.com"]
    }

    condition {
      test     = "StringEquals"
      variable = "AWS:SourceArn"
      values   = [aws_cloudfront_distribution.distribution.arn]
    }
  }
}

resource "aws_s3_bucket_policy" "policy" {
  bucket = aws_s3_bucket.bucket.id
  policy = data.aws_iam_policy_document.document.json
}

Running Terraform

Now, we can start provisioning the infrastructure.

# Initialize Terraform
terraform init

# Create a plan
terraform plan

# Apply the plan
terraform apply

This will take a few minutes to complete. Once it's done, we can manually upload our static website to the S3 bucket. You'll need to have the AWS CLI installed and configured and to copy the newly created S3 bucket name and CloudFront distribution ID.

# Remove the old files from S3 bucket
aws s3 rm s3://<bucket-name> --recursive

# Upload the website to the S3 bucket
aws s3 sync ./<dist-dir> s3://<bucket-name>

# Invalidate the CloudFront cache
aws cloudfront create-invalidation --distribution-id <distribution-id> --paths "/*"

This should make our website available at the domain name we specified.

We could've made uploading of website as part of the Terraform script, but it isn't the responsibility of Terraform to manage the content of the bucket. A better approach would be to use a CI/CD pipeline to automate the process of building and deploying the website, alongside keeping the infrastructure up to date.

Aside: Using external domain registrar

If you're using a domain registrar other than Route53, you'll need to create a CNAME record pointing to the CloudFront distribution and a CNAME record for the ACM certificate. For that you'll need to modify the Terraform script and remove the aws_route53_record resources.

Tear down

When we're done with the website, we can tear down the infrastructure by running:

terraform destroy

This will remove all the resources we created. For the S3 buckets and DynamoDB table you will have to do a manual cleanup.

Conclusion

In this article, we've learned how to deploy a static website to AWS S3 and CloudFront. This process can be simplified by using Vercel or Netlify, but their free plans have some limitations, so it's good to know how to host on your own.

I didn't cover the process of writing the terraform script in detail, but I hope that this article will give you a good starting point for your own projects. My recommendation is to use the Terraform AWS docs to figure out the available options. Also, manually setting up the infrastructure and then writing the Terraform scripts is a good way to learn how Terraform works and what is required to provision the resources.

If you have any questions or suggestions, feel free to contact me.

Scrutor — Assembly scanning and decoration extensions for Microsoft.Extensions.DependencyInjection

It reminded me of the Spring Framework in Java, which has a similar feature called @ComponentScan. It allows you to automatically register services in the container, which is very useful when you have a lot of services and don't want to register them manually. Although one of the downsides of this approach is that it can be hard to understand what services are registered in the container. And there is no quick way to find out just by looking at the code as in Program.cs or Startup.cs.

As a side note, in Spring Boot, you don't have to explicitly specify this annotation because it's automatically enabled when using @SpringBootApplication.

The course I took was from Nick Chapsas - From Zero to Hero: Dependency injection in .NET with C#, a well-known YouTube instructor with many years of experience building production APIs in .NET. I already finished his course about REST APIs in .NET, and I wasn't disappointed in the new course also.

The course covered some topics, including:

• Basic dependency injection and use cases
• How to use ServiceCollection and ServiceProvider with different lifetimes
• Resolving dependencies in different scenarios and projects
• Advanced topics, such as using ServiceDescriptor, Add vs. TryAdd, Open Generics, and deep dive into lifetime scopes, etc.
• Advanced techniques about custom scopes, decorators, avoiding service locator, and more
• Introduction to Scrutor and ways it can simplify service registration with potential pitfalls
• Creating your own minimal dependency injection framework by using a Dictionary

After completing the course, I got another certificate. Here it is:

It was a valuable investment to learn more about dependency injection in .NET and C#. If you have any questions or feedback, feel free to contact me.

The course I took was from Nick Chapsas - From Zero to Hero: REST APIs in .NET, a well-known YouTube instructor with many years of experience building production APIs in .NET. It taught me how to create a REST API end-to-end from scratch using the latest .NET 7, Dapper and other libraries. The course uses the C# (obviously), which is one of my favorites and the reasons for switching from Java and Node.js.

The REST API I built is a simple movie app that allows users to create, read, update and delete movies and ratings associated with a movie. It uses ASP.NET, Dapper for data access, PostgreSQL for database storage, and Swagger for documentation.

I've made a GitHub project that contains all the source code and instructions on how to run it locally. You can find it here: https://github.com/azdanov/MoviesApp

The course covered a bunch of topics, including:

• Basics of HTTP verbs and CRUD
• Building a RESTful API with data validation
• Authentication and authorization
• User-specific actions
• Filtering, sorting, pagination, versioning and swagger
• Building an SDK and consuming the API
• Minimal API

After completing the course, I got a certificate, nice surprise. Here it is:

I really enjoyed this course and can recommend it to anyone who wants to learn how to create REST APIs with .NET 7 and C#. If you have any questions or feedback, feel free to contact me.

Generally speaking, XACT_ABORT is a transaction option that determines whether the transaction is rolled back when certain additional errors occur which isn't turned on for backwards compatibility and NOCOUNT is a session option that determines whether the number of rows affected by a statement is returned to the client.

• XACT_ABORT - Good for avoiding unexpected behavior connected to rolling back a transactions
• NOCOUNT - Good for performance if the procedure is used in a loop by eliminating a network call

Here's a small example:

CREATE PROCEDURE insert
  @a int,
  @b int
AS
  SET XACT_ABORT, NOCOUNT ON;
  INSERT INTO table (a, b) VALUES (@a, @b);

Main resource that I used to learn more about these statements were from an article by Erland Sommarskog - Error and Transaction Handling in SQL Server.

For the theme, I decided to use Tailwind UI Spotlight, which is a paid theme offered as part of Tailwind UI that is a component library made with Tailwind CSS. I modified it a bit to fit my needs. And will probably continue to adjust it to my needs as I go.

Overall, I am very happy with the result. I like the way Next.js works and the theme looks fabulous. As a bonus, the project uses the same language everywhere, which is a big plus. It was a bit difficult when there were some Go-specific things while using Hugo.

Aside: React Hydration Error

While migrating I encountered this error:

Unhandled Runtime Error
Error: Hydration failed because the initial UI does not match what was rendered on the server.

See more info here: https://nextjs.org/docs/messages/react-hydration-error

Which documentation was very helpful in providing an example Invalid HTML may cause hydration mismatch such as div inside p..

Unknowingly I previously had a ul inside a p tag, which caused the error, it wasn't a valid HTML, yet it is allowed for backwards compatibility. An easy fix is to extract the ul from the p tag.

<!-- Error -->
<p>
  Text
  <ul>
    <li>First</li>
    <li>Second</li>
    <li>Third</li>
  </ul>
</p>
<!-- No error -->
<p>
  Text
</p>
<ul>
    <li>First</li>
    <li>Second</li>
    <li>Third</li>
  </ul>

I'm quite surprised and happy that Next.js helped me to learn more about writing correct HTML markup.

1. On-demand Self-Service. Can provision capabilities as needed without requiring human interaction with each service provider.
2. Broad Network Access. Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and workstations).
3. Resource Pooling. The provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, and network bandwidth.
4. Rapid Elasticity. Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time.
5. Measured Service. Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, reported and billed providing transparency for both the provider and consumer of the utilized service.

Summary:

1. On-demand Self-Service: Create and terminate resources through UI/CLI without human interaction.
2. Broad Network Access: Access resources over any network and any device using standard methods.
3. Resource Pooling: Due to combining of resources, the provider can offer resources at a lower cost.
4. Rapid Elasticity: Automatically scale resources up and down based on demand.
5. Measured Service: Usage of resources is measured and billed for what you consume.

In the 2nd chapter we are given an example of how to use HTTPS with spring boot:

1. Generate a certificate: openssl req -newkey rsa:2048 -x509 -keyout key.pem -out cert.pem -days 365

2. Convert it to 'p12' format: openssl pkcs12 -export -in cert.pem -inkey key.pem -out certificate.p12 -name "certificate"

3. Copy the generated 'certificate.p12' file inside 'src/main/resources' directory.

4. Update 'application.properties':

   server.ssl.key-store-type=PKCS12
   server.ssl.key-store=classpath:certificate.p12
   server.ssl.key-store-password=12345
   

One problem with this approach is that such certificates are not recognized by browsers, which will show an indicator that "this certificate is not trustworthy".

An easy workaround involves using a tool mkcert: https://github.com/FiloSottile/mkcert

With simple installation for macOS as an example:

# Install
brew install mkcert
brew install nss # if you use Firefox

# Setup local certificate authority (CA)
mkcert -install

# Generate certificate
mkcert -pkcs12 localhost 127.0.0.1

This will generate a new p12 ('localhost+2.p12') certificate for localhost and 127.0.0.1 in current directory that will be registered in local certificate authority (CA) which will keep browsers happy when doing local development.

The rest of the steps is the same, copy into 'src/main/resources' and update 'application.properties'.

Luckily, it's possible to define an initialization script that can help us out. The only inconvenience is in mounting a whole directory, since there is no easy way to mount a single file in Docker.

Here's how my docker-compose.yml looks:

• A directory pg-init-scripts that contains the initialization script is mounted into the docker container
• A custom environment variable POSTGRES_MULTIPLE_DATABASES is created with database names to be initialized

version: "3.8"
services:
  postgres:
    image: "postgres:13.2-alpine"
    container_name: postgres
    ports:
      - 5432:5432
    volumes:
      - ./pg-init-scripts:/docker-entrypoint-initdb.d
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=root
      - POSTGRES_MULTIPLE_DATABASES=accounts,bills,deposits

Now inside pg-init-scripts directory there is a script create_dbs.sh:

• Inside the script POSTGRES_USER is used as root user
• While POSTGRES_MULTIPLE_DATABASES is split on ,

#!/bin/bash

set -e
set -u

function create_user_and_database() {
    local database=$1
    echo "Creating user and database '$database'"
    psql -v ON_ERROR_STOP=1 --username "$POSTGRES_USER" <<-EOSQL
      CREATE USER $database;
      CREATE DATABASE $database;
      GRANT ALL PRIVILEGES ON DATABASE $database TO $database;
EOSQL
}

if [ -n "$POSTGRES_MULTIPLE_DATABASES" ]; then
    echo "Creating databases: $POSTGRES_MULTIPLE_DATABASES"
    for db in $(echo "$POSTGRES_MULTIPLE_DATABASES" | tr ',' ' '); do
        create_user_and_database "$db"
    done
    echo "Multiple databases created"
fi

Try it out and see if it helps you.

Here's the reference to PostgreSQL Docker image: https://hub.docker.com/_/postgres/ with more information about initialization scripts.

After trying out Go for hobby projects I've been intrigued with a static site generator Hugo and decided to try it with a custom theme hello-friend-ng.

The end result was very satisfying. It was easy to setup, build time is lightning fast and thus far, I assume, maintenance will be also minimal.

Let's see if I will focus more time on writing and less on resolving merge conflicts or npm updates for my website. 😄

Spring offers amazing caching utilities. Throw in some @EnableCaching, @Cacheable, @CacheEvict on and the results will be cached. You can read more about Spring Caching on https://www.baeldung.com/spring-cache-tutorial.

The problem that I encountered was figuring out how to use EhCache 3 (org.ehcache) and inside Spring Boot with Java Based Configuration. A lot of how-to guides were either showing ehcache.xml or an older EhCache 2 (net.sf.ehcache) configuration convention.

Let's see how to do configure EhCache 3 programmatically.

Setup

First step is adding required dependencies to the project (Maven/Gradle):

<!-- pom.xml -->
<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-cache</artifactId>
    </dependency>
    <dependency>
        <groupId>org.ehcache</groupId>
        <artifactId>ehcache</artifactId>
        <version>3.9.0</version>
    </dependency>
    <dependency>
        <groupId>javax.cache</groupId>
        <artifactId>cache-api</artifactId>
        <version>1.1.1</version>
    </dependency>
</dependencies>

or

// build.gradle
dependencies {
  implementation 'org.springframework.boot:spring-boot-starter-cache'
  implementation 'org.ehcache:ehcache:3.9.0'
  implementation 'javax.cache:cache-api:1.1.1'
}

Configuration

Next step would be creating a configuration class:

package org.js.azdanov.ehcacheconfigdemo.configuration;

import org.ehcache.config.builders.CacheConfigurationBuilder;
import org.ehcache.config.builders.ExpiryPolicyBuilder;
import org.ehcache.config.builders.ResourcePoolsBuilder;
import org.ehcache.jsr107.Eh107Configuration;
import org.springframework.boot.autoconfigure.cache.JCacheManagerCustomizer;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import javax.cache.Cache;
import javax.cache.CacheManager;
import javax.cache.configuration.CacheEntryListenerConfiguration;
import javax.cache.configuration.Factory;
import javax.cache.configuration.FactoryBuilder;
import javax.cache.configuration.MutableCacheEntryListenerConfiguration;
import javax.cache.event.CacheEntryEventFilter;
import java.time.Duration;

@EnableCaching
@Configuration
public class CacheConfiguration {
    private static final Factory<? extends CacheEntryEventFilter<Integer, Integer>> NO_FILTER = null;
    private static final boolean IS_OLD_VALUE_REQUIRED = false;
    private static final boolean IS_SYNCHRONOUS = true;

    // (1)
    private final javax.cache.configuration.Configuration<Integer, Integer> jcacheConfiguration =
        Eh107Configuration.fromEhcacheCacheConfiguration(
            CacheConfigurationBuilder
                .newCacheConfigurationBuilder(Integer.class, Integer.class, ResourcePoolsBuilder.heap(100))
                .withExpiry(ExpiryPolicyBuilder.timeToLiveExpiration(Duration.ofMinutes(5)))
                .build()
        );

    CacheEntryListenerConfiguration<Integer, Integer> listenerConfiguration =
        new MutableCacheEntryListenerConfiguration<>(
            FactoryBuilder.factoryOf(CacheListener.class),
            NO_FILTER,
            IS_OLD_VALUE_REQUIRED,
            IS_SYNCHRONOUS);

    // (2)
    @Bean
    public JCacheManagerCustomizer cacheManagerCustomizer() {
        return cm -> {
            createCache(cm, "worker1");
            createCache(cm, "worker2");
        };
    }

    private void createCache(CacheManager cm, String cacheName) {
        Cache<Integer, Integer> cache = cm.getCache(cacheName);
        if (cache == null) {
            cm.createCache(cacheName, jcacheConfiguration)
                .registerCacheEntryListener(listenerConfiguration);
        }
    }
}

1. Here's the configuration itself, which can be modified to read the values from an application.properties file or any other resource.
2. Inside JCacheManagerCustomizer bean the individual cache creation happens.

I've added some extras where a cache event listener will be registered for each created cache.

Here's how a listener class might look like:

package org.js.azdanov.ehcacheconfigdemo.configuration;

import lombok.extern.slf4j.Slf4j;

import javax.cache.event.CacheEntryCreatedListener;
import javax.cache.event.CacheEntryEvent;

@Slf4j
public class CacheListener implements CacheEntryCreatedListener<Integer, Integer> {

    @Override
    public void onCreated(final Iterable<CacheEntryEvent<? extends Integer, ? extends Integer>> cacheEntryEvents) {
        for (CacheEntryEvent<? extends Integer, ? extends Integer> entryEvent : cacheEntryEvents) {
            log.info("Cached key: {}, with value: {}", entryEvent.getKey(), entryEvent.getValue());
        }
    }
}

Now for the last step let's mark methods inside a service with previously defined cache names:

package org.js.azdanov.ehcacheconfigdemo.service;

import lombok.RequiredArgsConstructor;
import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

import java.util.Random;

@RequiredArgsConstructor
@Service
public class WorkerService {

    private final Random random;

    @Cacheable(value = "worker1", key = "#bound")
    public int getFirstWork(int bound) {
        return random.nextInt(bound);
    }

    @Cacheable(value = "worker2", key = "#bound")
    public int getSecondWork(int bound) {
        return random.nextInt(bound);
    }
}

Last step

It's time to see if this works or not with a simple sanity-check 'test':

package org.js.azdanov.ehcacheconfigdemo;

import lombok.extern.slf4j.Slf4j;
import org.js.azdanov.ehcacheconfigdemo.service.WorkerService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.context.event.ApplicationReadyEvent;
import org.springframework.cache.annotation.EnableCaching;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.event.EventListener;

import java.util.Random;

@Slf4j
@EnableCaching
@Configuration
@SpringBootApplication
public class EhcacheConfigDemoApplication {

    @Autowired
    WorkerService workerService;

    public static void main(String[] args) {
        SpringApplication.run(EhcacheConfigDemoApplication.class, args);
    }

    @Bean
    Random random() {
        return new Random();
    }

    @EventListener(ApplicationReadyEvent.class)
    public void afterStartup() {
        log.info("Running!");
        log.info("getFirstWork(100): {}", workerService.getFirstWork(100));
        log.info("getFirstWork(50): {}", workerService.getFirstWork(50));
        log.info("getFirstWork(100): {}", workerService.getFirstWork(100));
        log.info("getFirstWork(50): {}", workerService.getFirstWork(50));
        log.info("getSecondWork(100): {}", workerService.getSecondWork(100));
        log.info("getSecondWork(50): {}", workerService.getSecondWork(50));
        log.info("getSecondWork(100): {}", workerService.getSecondWork(100));
        log.info("getSecondWork(50): {}", workerService.getSecondWork(50));
    }
}

This is the result I got in the terminal:

Running!
Cached key: 100, with value: 8
getFirstWork(100): 8
Cached key: 50, with value: 29
getFirstWork(50): 29
getFirstWork(100): 8
getFirstWork(50): 29
Cached key: 100, with value: 6
getSecondWork(100): 6
Cached key: 50, with value: 33
getSecondWork(50): 33
getSecondWork(100): 6
getSecondWork(50): 33

Seems to be working 🙂

Sadly this led to neglecting my website for a while... Not any longer.

I have decided to freshen up my blog and moving away from Gatsby. The main reason is simplicity, because 11ty offers simple solution for a simple blog without the complexity of GraphQL or React.

Special thanks to cramforce for creating eleventy-high-performance-blog starter kit.

After ~3 years of studying, building, learning and pushing forward every day, I've decided that my knowledge has reached the level adequate for working in professional development.

I've started sending out my résumé to local companies, and to my delight, I didn't have to wait long. Several offered a chance for an interview by building a small test project. The interviews went well, and I had to choose which role and technology stack to work with.

In the end, the company that I'm working for is using a Java and Vue.js stack for their projects. My role is as a Full Stack Developer.

Encouragements

For those who are studying either in a university or on their own, the only advice I can give is not to give up. Sooner or later you will reach your set goals. And the only way there is one step at a time.

I'll be honest with you, I suck at algorithms and data structures. Trying to figure the proper approach is a difficult and brain-frying endeavor. And don't get me started on the recursion, which makes everything 10x more challenging to reason about.

GitHub for PHP Interview Exercises

Luckily things get easier with experience, repetition and simple know-how. Old challenges that seemed insurmountable become easier to reason about. That's why I love to return and rehearse some basic algorithms and data structures from time to time. It just makes you feel better when you finally understand how a linked list works, or a tree traversal is accomplished.

Test-driven development

Wikipedia for TDD

This time I wanted to try a TDD approach (more or less.) By writing a test first, and the implementation with possible refactor it is easier to solve a problem. Instead of staring into an empty code editor you start writing and thinking about the problem.

To write a proper test you need to:

• Understand the problem (What is my goal)
• Explore test cases (Find proper input and output)

And already half of the problem is solved. Now what rests is to create a solution.

What we can do now is:

• Break the problem down into smaller chunks & solve it
• Simplify once all the test are green
• Look & Refactor for performance (Remember that premature optimization is bad)

Problem-solving is an important skill to hone for a developer. By practicing on smaller chunks of code it is easier to gauge one's level and simply stay motivated.

Happy coding!

I would like to start a collection of useful videos that I've found online.

Although they are centered around Laravel, the ideas are easy to adopt in any other framework.

Here goes the first batch!

Matt Stauffer - Patterns that pay off

Jeffrey Way - Slay the Beast: Nine considerations to get your app back on track

Adam Wathan - Resisting Complexity

Adam Wathan - Cruddy by design

For a while I've been curious about Go and its relevance to Web development.

I've heard a lot of good things about the performance, its rich standard library and how it has become a first language for many PHP and Node developers.

So let's have a go at Go!

CLI App

You can view the code hosted on GitHub at azdanov/images.

Here's the finished version, since this is not a tutorial I will not explain what each part of the code does.

package main

import (
	"bufio"
	"encoding/base64"
	"fmt"
	"log"
	"mime"
	"os"
	"path/filepath"
)

func main() {
	if len(os.Args) == 1 {
		log.Fatal("No arguments specified.")
	}

	image := os.Args[1]

	file, err := os.Open(image)
	if os.IsNotExist(err) {
		log.Fatal("File not found.")
	}

	mimeType := mime.TypeByExtension(filepath.Ext(os.Args[1]))

	supported := []string{
		"image/svg",
		"image/svg+xml",
		"image/gif",
		"image/jpeg",
		"image/png",
	}

	if !inSlice(mimeType, supported) {
		log.Fatal("Image type not supported.")
	}

	info, _ := file.Stat()
	buf := make([]byte, info.Size())

	fReader := bufio.NewReader(file)
	_, _ = fReader.Read(buf)

	data := fmt.Sprintf(
		"data:%s;base64,%s",
		mimeType,
		base64.StdEncoding.EncodeToString(buf)
	)

	fmt.Print(data)
}

func inSlice(a string, list []string) bool {
	for _, b := range list {
		if a == b {
			return true
		}
	}
	return false
}

Original was made in PHP:

<?php

declare(strict_types=1);

array_shift($argv);
$image = array_shift($argv);

if (!$image) {
    fwrite(STDERR, 'No arguments specified.');
    die(1);
}
if (!file_exists($image)) {
    fwrite(STDERR, 'File not found.');
    die(1);
}

$mime = mime_content_type($image);

$supported = ['image/svg', 'image/svg+xml', 'image/gif', 'image/jpeg', 'image/png'];
if (!in_array($mime, $supported)) {
    fwrite(STDERR, 'Image type not supported.');
    die(1);
}

if ($mime === 'image/svg') {
    $mime .= '+xml';
}

$data = file_get_contents($image);

die("data:${mime};base64," . base64_encode($data));

And for fun here's Node:

const { existsSync, readFileSync } = require("fs");
const { extname } = require("path");

if (process.argv.length == 2) {
  process.stderr.write("No arguments specified.");
  process.exit(1);
}

const image = process.argv[2];

if (!existsSync(image)) {
  process.stderr.write("File not found.");
  process.exit(1);
}

const extension = extname(image).substr(1);
const supported = ["image/svg+xml", "image/gif", "image/jpeg", "image/png"];
const mime = supported.find((s) => s.includes(extension));

if (!mime) {
  process.stderr.write("Image type not supported.");
  process.exit(1);
}

const data = readFileSync(image);
process.stdout.write(`data:${mime};base64,${data.toString("Base64")}`);

Benchmark

I was curious how much Go would outperform PHP and Node, and was pleasantly surprised.

hyperfine was used for the benchmarks.

PHP (PHP 7.3.3)	Go (go1.12.1 darwin/amd64)	Node (v11.13.0)
User: 20.6ms, System: 9.8ms 74 runs	User: 5.1ms, System: 2.2ms 286 runs	User: 72.9ms, System: 18.7ms 30 runs

Finish

I didn't expect Node to be that slow compared to PHP. I guess PHP is performant enough for most Web related tasks.

So in conclusion Go is really an interesting language, there is a good reason it's called the C language of the web.

In Tailwind CSS there is an amazing utility for adding padding and margin which makes spatial positioning of element a joy.

While working with Bulma I've started to miss previously provided handy functionality.

This snippet should remedy this:

$sizeUnit: rem;
$marginKey: "m";
$paddingKey: "p";
$separator: "-";
$sizes: (
  ("0", 0),
  ("1", 0.125),
  ("2", 0.25),
  ("3", 0.5),
  ("4", 1),
  ("5", 2),
  ("6", 4),
  ("7", 5.5)
);
$positions: (("t", "top"), ("r", "right"), ("b", "bottom"), ("l", "left"));

@function sizeUnitValue($key, $value) {
  @return if($key == "none", 0, $value + $sizeUnit);
}

@each $size in $sizes {
  $sizeKey: nth($size, 1);
  $sizeValue: nth($size, 2);
  .#{$marginKey}#{$separator}#{$sizeKey} {
    margin: sizeUnitValue($sizeKey, $sizeValue);
  }
  .#{$paddingKey}#{$separator}#{$sizeKey} {
    padding: sizeUnitValue($sizeKey, $sizeValue);
  }
  @each $position in $positions {
    $posKey: nth($position, 1);
    $posValue: nth($position, 2);
    .#{$marginKey}#{$posKey}#{$separator}#{$sizeKey} {
      margin-#{$posValue}: sizeUnitValue($sizeKey, $sizeValue);
    }
    .#{$paddingKey}#{$posKey}#{$separator}#{$sizeKey} {
      padding-#{$posValue}: sizeUnitValue($sizeKey, $sizeValue);
    }
  }
}

This will generate:

/* ... */
.m-1 {
  margin: 0.125rem;
}
.p-1 {
  padding: 0.125rem;
}
.mt-1 {
  margin-top: 0.125rem;
}
.pt-1 {
  padding-top: 0.125rem;
}
.mr-1 {
  margin-right: 0.125rem;
}
.pr-1 {
  padding-right: 0.125rem;
}
.mb-1 {
  margin-bottom: 0.125rem;
}
.pb-1 {
  padding-bottom: 0.125rem;
}
.ml-1 {
  margin-left: 0.125rem;
}
.pl-1 {
  padding-left: 0.125rem;
}
/* ... */

Quite useful.

For example there is a dashboard for a fictitious logistics company, they have a multitude of workers, each frequently visiting the website.

But there is a problem. The summary page that is found on the / route needs to execute a heavy query on each visit, which slows everything down. Not to mention the lack of reusability when embedding the summary SQL code inside the app.

-- Employee Leaderboard
SELECT
  concat(e.firstname, ' ', e.lastname) AS name,
  sum(od.price * od.quantity) AS amount
FROM
  employee AS e
  INNER JOIN order AS o ON o.employeeid = e.id
  INNER JOIN orderdetail AS od ON od.orderid = o.id
GROUP BY
  e.id
ORDER BY
  amount DESC
LIMIT
  5;

-- Customer Leaderboard
SELECT
  c.company AS name,
  sum(od.price * od.quantity) AS amount
FROM
  customer AS c
  INNER JOIN order AS o ON o.customerid = c.id
  INNER JOIN orderdetail AS od ON od.orderid = o.id
GROUP BY
  c.id
ORDER BY
  amount DESC
LIMIT
  5;

-- Product Leaderboard
SELECT
  p.product AS name, sum(od.price * od.quantity) AS amount
FROM
  orderdetail AS od
  INNER JOIN order AS o ON o.id = od.orderid
  INNER JOIN product AS p ON p.id = od.productid
GROUP BY
  p.id
ORDER BY
  amount DESC
LIMIT
  5;

-- Recent Orders
SELECT
  o.id,
  concat(e.firstname, ' ', e.lastname) AS employee,
  c.company AS customer,
  o.date,
  sum(od.price * od.quantity) AS amount
FROM
  order AS o
  INNER JOIN orderdetail AS od ON od.orderid = o.id
  INNER JOIN employee AS e ON e.id = o.employeeid
  INNER JOIN customer AS c ON c.id = o.customerid
WHERE
  o.date IS NOT NULL
GROUP BY
  o.id, e.firstname, e.lastname, c.company
ORDER BY
  o.date DESC
LIMIT
  5;

-- Product Reorder list
SELECT
  product AS name, reorderthreshold, unitsinstock, unitsordered
FROM
  product
WHERE
  (unitsinstock + unitsordered) < reorderthreshold;

Let's move this query inside the database as a MATERIALIZED VIEW.

Refactor to PostgreSQL Materialized View

PostgreSQL natively supports MATERIALIZED VIEW, which will simplify things a lot. The queries stay basically the same, so it's a simple refactor.

One problem one might encounter is a need to manually refresh the views, which can be solved with TRIGGER.

-- Employee Leaderboard
CREATE MATERIALIZED VIEW mv_employeeleaderboard
AS
SELECT
  concat(e.firstname, ' ', e.lastname) AS name,
  sum(od.price * od.quantity) AS amount
FROM
  employee AS e
  INNER JOIN order AS o ON o.employeeid = e.id
  INNER JOIN orderdetail AS od ON o.id = od.orderid
  GROUP BY
    e.id
  ORDER BY
    amount DESC
  LIMIT 5;

-- Customer Leaderboard
CREATE MATERIALIZED VIEW mv_customerleaderboard
AS
SELECT
  c.company AS name,
  sum(od.price * od.quantity) AS amount
FROM
  customer AS c
  INNER JOIN order AS o ON o.customerid = c.id
  INNER JOIN orderdetail AS od ON o.id = od.orderid
  GROUP BY
    c.id
  ORDER BY
    amount DESC
  LIMIT 5;

-- Product Leaderboard
CREATE MATERIALIZED VIEW mv_productleaderboard
AS
SELECT
  p.product AS name,
  sum(od.price * od.quantity) AS amount
FROM
  orderdetail AS od
  INNER JOIN order AS o ON od.orderid = o.id
  INNER JOIN product AS p ON od.productid = p.id
  GROUP BY
    p.id
  ORDER BY
    amount DESC
  LIMIT 5;

-- Recent Orders
CREATE VIEW mv_recentorders
AS
SELECT
  o.id,
  concat(e.firstname, ' ', e.lastname) AS employee,
  c.company AS customer,
  o.date,
  sum(od.price * od.quantity) AS subtotal
FROM
  order AS o
  INNER JOIN orderdetail AS od ON od.orderid = o.id
  INNER JOIN employee AS e ON o.employeeid = e.id
  INNER JOIN customer AS c ON o.customerid = c.id
  WHERE
    o.date IS NOT NULL
  GROUP BY
    o.id,
    e.firstname,
    e.lastname,
    c.company
  ORDER BY
    o.date DESC
  LIMIT 5;

-- Product Reorder list (VIEW is used to keep it up-to-date)
CREATE VIEW v_reorderlist
SELECT
  product AS name, reorderthreshold, unitsinstock, unitsordered
FROM
  product
WHERE
  (unitsinstock + unitsordered) < reorderthreshold;

Now let's create some TRIGGERs and a FUNCTION that will refresh created views on order update or delete.

CREATE OR REPLACE FUNCTION refresh_dashboard()
  RETURNS TRIGGER
AS $$
BEGIN
  REFRESH MATERIALIZED VIEW mv_employeeleaderboard;
  REFRESH MATERIALIZED VIEW mv_customerleaderboard;
  REFRESH MATERIALIZED VIEW mv_productleaderboard;
  REFRESH MATERIALIZED VIEW mv_recentorders;
  RETURN new;
END;
$$
LANGUAGE 'plpgsql';

CREATE TRIGGER dashboard_refresh_for_order_update
AFTER UPDATE
  ON order FOR EACH ROW EXECUTE PROCEDURE refresh_dashboard();

CREATE TRIGGER dashboard_refresh_for_order_delete
AFTER DELETE
  ON order FOR EACH ROW EXECUTE PROCEDURE refresh_dashboard();

These statements can be entered directly via a terminal into a database, or even better inserted by a migration.

It is easy to revert introduced changes with these statements.

DROP MATERIALIZED VIEW mv_employeeleaderboard;
DROP MATERIALIZED VIEW mv_customerleaderboard;
DROP MATERIALIZED VIEW mv_productleaderboard;
DROP MATERIALIZED VIEW mv_recentorders;
DROP VIEW v_reorderlist;

DROP TRIGGER dashboard_refresh_for_order_update ON order;
DROP TRIGGER dashboard_refresh_for_order_delete ON order;

Refactor to MySQL Materialized View Hack

Sadly MySQL doesn't have MATERIALIZED VIEW, what we could do is create a specialized table only used for the dashboard and manually update it's data.

First step is to encapsulate each query inside of a dedicated VIEW.

-- Employee Leaderboard
CREATE OR REPLACE VIEW v_employeeleaderboard
AS (
  SELECT
    concat(e.firstname, ' ', e.lastname) AS name,
    sum(od.price * od.quantity) AS amount
  FROM
    employee AS e
  INNER JOIN order AS o ON o.employeeid = e.id
  INNER JOIN orderdetail AS od ON o.id = od.orderid
  GROUP BY
    e.id
  ORDER BY
    amount DESC
  LIMIT 5);

-- Customer Leaderboard
CREATE OR REPLACE VIEW v_customerleaderboard
AS (
  SELECT
    c.company AS name,
    SUM(od.price * od.quantity) AS amount
  FROM
    customer AS c
  INNER JOIN order AS o ON o.customerid = c.id
  INNER JOIN orderdetail AS od ON o.id = od.orderid
  GROUP BY
    c.id
  ORDER BY
    amount DESC
  LIMIT 5);

-- Product Leaderboard
CREATE OR REPLACE VIEW v_productleaderboard
AS (
  SELECT
    p.product AS name,
    sum(od.price * od.quantity) AS amount
  FROM
    orderdetail AS od
  INNER JOIN order AS o ON od.orderid = o.id
  INNER JOIN product AS p ON od.productid = p.id
  GROUP BY
    p.id
  ORDER BY
    amount DESC
  LIMIT 5);

-- Recent Orders
CREATE OR REPLACE VIEW v_recentorders
AS (
  SELECT
    o.id,
    concat(e.firstname, ' ', e.lastname) AS employee,
    c.company AS customer,
    o.date,
    sum(od.price * od.quantity) AS amount
  FROM
    order AS o
  INNER JOIN orderdetail AS od ON od.orderid = o.id
  INNER JOIN employee AS e ON o.employeeid = e.id
  INNER JOIN customer AS c ON o.customerid = c.id
  WHERE
    o.date IS NOT NULL
  GROUP BY
    o.id,
    e.firstname,
    e.lastname,
    c.company
  ORDER BY
    o.date DESC
  LIMIT 5);

-- Product Reorder list (VIEW is used to keep it up-to-date)
CREATE OR REPLACE VIEW v_reorderlist
SELECT
  product AS name, reorderthreshold, unitsinstock, unitsordered
FROM
  product
WHERE
  (unitsinstock + unitsordered) < reorderthreshold;

Next is to create a TABLE that will imitate MATERIALIZED VIEW.

CREATE TABLE mv_employeeleaderboard AS
SELECT * FROM v_employeeleaderboard;

CREATE TABLE mv_customerleaderboard AS
SELECT * FROM v_customerleaderboard;

CREATE TABLE mv_productleaderboard AS
SELECT * FROM v_productleaderboard;

CREATE TABLE mv_recentorders AS
SELECT * FROM v_recentorders;

CREATE TRIGGER dashboard_refresh_for_order_update
AFTER UPDATE
  ON order FOR EACH ROW
BEGIN
  DELETE FROM mv_employeeleaderboard;
  DELETE FROM mv_customerleaderboard;
  DELETE FROM mv_productleaderboard;
  DELETE FROM mv_recentorders;

  INSERT INTO mv_employeeleaderboard
  SELECT * FROM v_employeeleaderboard;

  INSERT INTO mv_customerleaderboard
  SELECT * FROM v_customerleaderboard;

  INSERT INTO mv_productleaderboard
  SELECT * FROM v_productleaderboard;

  INSERT INTO mv_recentorders
  SELECT * FROM v_recentorders;
END;

CREATE TRIGGER dashboard_refresh_for_delete
AFTER DELETE
  ON order FOR EACH ROW
BEGIN
  DELETE FROM mv_employeeleaderboard;
  DELETE FROM mv_customerleaderboard;
  DELETE FROM mv_productleaderboard;
  DELETE FROM mv_recentorders;

  INSERT INTO mv_employeeleaderboard
  SELECT * FROM v_employeeleaderboard;

  INSERT INTO mv_customerleaderboard
  SELECT * FROM v_customerleaderboard;

  INSERT INTO mv_productleaderboard
  SELECT * FROM v_productleaderboard;

  INSERT INTO mv_recentorders
  SELECT * FROM v_recentorders;
END;

And to revert the changes.

DROP VIEW v_employeeleaderboard;
DROP VIEW v_customerleaderboard;
DROP VIEW v_productleaderboard;
DROP VIEW v_recentorders;
DROP VIEW v_reorderlist;

DROP TABLE IF EXISTS mv_customerleaderboard;
DROP TABLE IF EXISTS mv_employeeleaderboard;
DROP TABLE IF EXISTS mv_productleaderboard;
DROP TABLE IF EXISTS mv_recentorders;

DROP TRIGGER IF EXISTS dashboard_refresh_for_order_update;
DROP TRIGGER IF EXISTS dashboard_refresh_for_order_delete;

Final

In the end this will significantly speedup the dashboard loading. In a future project I will create a logistics mock application that will feature this enhancement.

PS: To use the views in your application code:

SELECT * FROM mv_employeeleaderboard;
SELECT * FROM mv_customerleaderboard;
SELECT * FROM mv_productleaderboard;
SELECT * FROM mv_recentorders;
SELECT * FROM v_reorderlist;

One day I stumbled upon an interesting question: "How to generate a list of hashes for all the page images inside the browser?". It got me intrigued about how versatile JavaScript actually is. Thus here are the result of that endeavor.

Prelude

I will be using https://picsum.photos/ since there are no CORS issues. For any other website an extension/flag might be required to disable CORS protection inside the browser.

For the MD5 library there are many choices, through trial and error I've selected https://github.com/emn178/js-md5/. The most common issue with others was lack of support for ArrayBuffer which resulted in incorrect hash values.

Loading an External Package

While this might sound daunting at first, it's not that bad. Most packages can be executed inside the browser without any compilation via babel/webpack.

Inside the browser console it is done by creating a script element and adding a source to the script file. In our case the source will be https://cdn.jsdelivr.net/npm/js-md5@0.7.3/src/md5.min.js.

var script = document.createElement("script");
script.src = "https://cdn.jsdelivr.net/npm/js-md5@0.7.3/src/md5.min.js";

document.querySelector("head").appendChild(script);

If there were no CORS issues then an md5 function will be available inside the terminal. Note: this can also be used to require jQuery or similar libraries to do some DOM heavy lifting.

Getting the Sources

Next it is necessary to find all the image sources on the page. With ES6 it is a trivial task.

var imgs = [...document.querySelectorAll("img")];
var imgSrcs = imgs.map((i) => i.src);

Preparing to Fetch

Now that imgSrcs array has all the sources a function is needed to fetch each source, and convert it into a Blob. Also this blog object must be converted into an ArrayBuffer. Most modern browsers have FileReader API that facilitates working with Files and Blobs. And last but not least Fetch is a modern replacement for XMLHttpRequest.

var getData = (url) =>
  fetch(url)
    .then((response) => response.blob())
    .then(
      (blob) =>
        new Promise((resolve, reject) => {
          const reader = new FileReader();
          reader.onloadend = () => resolve(reader.result);
          reader.onerror = reject;
          reader.readAsArrayBuffer(blob);
        }),
    );

This function will be used later to generate a list of Promises. Inside the second then block a new Promise is created. It is necessary because reader.readAsArrayBuffer is an asynchronous operation, that triggers onloadend or onerror after a certain period of time.

Making Promises

It is time to send out the request for each image.

var promises = Promise.all(imgSrcs.map(getData));

Promise.all helps in reducing overall delay by sending out the fetch requests simultaneously. Of course it might fail if one image is corrupt.

Creating Hashes

Final step would be to iterate over the promises array, convert each ArrayBuffer into md5 and print out the results.

promises
  .then((buffers) => buffers.map(md5))
  .then((hashes) => console.log(hashes));

Final

There are many other things that could be tried. Another hashing algorithm such as SHA256. Web Crypto API could be used instead of injecting an external library (Only for SHA). And what about error handling, since Promise.all will throw an error if any of the promises fail.

Code:

// Get md5 library
var md5 = document.createElement("script");
md5.src = "https://cdn.jsdelivr.net/npm/js-md5@0.7.3/src/md5.min.js";
document.querySelector("head").appendChild(md5);

// Prepare function to get image blobs
var getData = (url) =>
  fetch(url)
    .then((response) => response.blob())
    .then(
      (blob) =>
        new Promise((resolve, reject) => {
          const reader = new FileReader();
          reader.onloadend = () => resolve(reader.result);
          reader.onerror = reject;
          reader.readAsArrayBuffer(blob);
        }),
    );

// Get image sources
var imgSrcs = [...document.querySelectorAll("img")].map((i) => i.src);

// Load images
var promises = Promise.all(imgSrcs.map(getData));

// Calculate hashes
promises
  .then((buffers) => buffers.map(md5))
  .then((hashes) => console.log(hashes));

In the end I was satisfied with the results, because modern JavaScript is a powerful tool to know and use.

When doing a recent project I decided to add offline capabilities. This was a dictionary app that relied on internet connectivity, and each new request was expensive time-wise. Luckily, Create React App comes with an already configured Service Worker that eases turning a React project into a Progressive Web App.

Enabling the Service Worker

Inside src/index.js change serviceWorker.unregister() to serviceWorker.register().

This will allow service worker to cache external resources and app files. Which for many use cases is enough.

Note that by default service workers will update itself only on total page restart, which means closing all tabs on a device and opening the app again. This can be confusing.

Updating the Service Worker

Sometimes updating the app is critical to its functionality. On a mobile device a hidden open tab can prevent a service worker update. Which can lead to a lot of frustration for both the developer and user.

The easiest way around this is to trigger a force update inside a service worker. This can be potentially dangerous if the user enters any data inside the app, so keep that in mind.

Currently, there's no way to add a config option inside webpack to enable this.

Let's do it manually, using fs module from node.

After running npm build a build/service-worker.js will be created:

importScripts(
  "https://storage.googleapis.com/workbox-cdn/releases/3.6.3/workbox-sw.js",
);

importScripts("/precache-manifest.1627229ec6fca0a0029e621a9027a2fd.js");

workbox.clientsClaim();

self.__precacheManifest = [].concat(self.__precacheManifest || []);
workbox.precaching.suppressWarnings();
workbox.precaching.precacheAndRoute(self.__precacheManifest, {});

workbox.routing.registerNavigationRoute("/index.html", {
  blacklist: [/^\/_/, /\/[^\/]+\.[^\/]+$/],
});

To enable force an update self.skipWaiting() needs to be added here.

And node is well suited for this job.

1. Create a file modifyServiceWorker.js

const fs = require("fs");

fs.readFile("build/service-worker.js", "utf8", (err, data) => {
  if (err) return console.error(err);

  const snippet = `
self.addEventListener('install', event => {
  self.skipWaiting();
});
  `;

  const result = data.replace(
    "workbox.clientsClaim();",
    `workbox.clientsClaim();\n${snippet}`,
  );

  fs.writeFile("build/service-worker.js", result, "utf8", (readError) => {
    if (readError) return console.log(readError);
  });
});

2. Modify the npm build script

{
  "scripts": {
    "build": "react-scripts build && node modifyServiceWorker.js"
  }
}

This will skip the waiting lifecycle of service worker and update it immediately, which can be good and bad. This depends on the app itself.

Prompting the user to update

For better user experience would be to ask the user if they are ready for an update.

This is slightly more complicated, but in the end more considerate.

To achieve such behavior we need to display a button, that will send a message to the service worker on clicking it. Here is a general idea:

1. Adjust modifyServiceWorker.js to listen for a message

const fs = require("fs");

fs.readFile("build/service-worker.js", "utf8", (err, data) => {
  if (err) return console.error(err);

  const snippet = `
  addEventListener('message', messageEvent => {
    if (messageEvent.data === 'skipWaiting') return skipWaiting();
  });
  `;

  const result = data.replace(
    "workbox.clientsClaim();",
    `workbox.clientsClaim();\n${snippet}`,
  );

  fs.writeFile("build/service-worker.js", result, "utf8", (readError) => {
    if (readError) return console.log(readError);
  });
});