isopov

Benchmarking memory throughput of 14900K CPU

Fri, 29 Nov 2024 11:57:39 +0300

In previous post I benchmark a heavily CPU-bound code. It may not seem so, however it appears to be memory bound. I’ve read somewhere that 14900 CPU efficient cores scale worse on memory accesses, than on just cracking numbers. So I wrote the following:

@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@Fork(3)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 3, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
public class ThreadsBenchmark {

 @Param("default")
 public String threads;


 @Benchmark
 public BigInteger bench() {
 var result = BigInteger.ONE;
 for (int i = 1; i <= 2048; i++) {
 result = result.multiply(BigInteger.valueOf(i));
 }
 return result;
 }

 public static void main(String[] args) throws RunnerException {
 var results = new ArrayList<RunResult>();
 for (int i = 1; i <= 36; i++) {
 Options opt = new OptionsBuilder()
 .include(".*" + ThreadsBenchmark.class.getSimpleName() + ".*")
 .addProfiler(GCProfiler.class)
 .verbosity(VerboseMode.SILENT)
 .param("threads", String.valueOf(i))
 .threads(i)
 .build();
 results.addAll(new Runner(opt).run());
 System.out.println("Finished " + i);
 }

 OutputFormatFactory.createFormatInstance(System.out, Defaults.VERBOSITY).endRun(results);
 }
}

And got the following results:

Benchmarking 14900K CPU

Mon, 25 Nov 2024 22:42:39 +0300

So, I got bored with waiting for tests of my laptop and built myself a new desktop for the first time in 10 years. I’ve built it around 14900K CPU and no separate GPU at all. For the OS I use Ubuntu 24.04. And I wanted to look closer to the differences between cores in this CPU. It has 8 performance cores and 16 efficient cores. So I wrote this benchmark to test them.

Go concurrent sort of the same slice

Mon, 05 Aug 2024 11:55:49 +0300

Recently I’ve stumbled upon a broken cache in my application written in Go. Some slices of structs got from database were cached in memory. In the hindsight the problem is obvious - every consumer of this cache sorted these slices independently. And since slices were cached in some cases they were sorting one and the same slice concurrently. Someone may suggest that sorting should be done in the database with sort by, but even in the simplest case where it can be done on this level simply adding SORT BY clause it may be not desirable. Usually application layer is scaled much easier than the storage layer. Only some newest databases introduce their own scalable computational nodes separate from the storage nodes. Traditionally you may want to make your queries as lightweight as possible and move some CPU-bound postprocessing to application layer. Also, you don’t want to query your database for one and the same information over and over. So caches are introduced to the application layer. And here we go with DAO -> Cache -> Provider/Manager.

Go GC, generational hypothesis and throughput

Mon, 18 Mar 2024 10:10:42 +0300

In the previous post I investigated effect of old generation objects on Java garbage collectors. Here I want to run similar test for Go language. It is a bit less interesting since there is only one option, but nevertheless some useful insights can be taken out. Like how much RAM to give for GC breathing. Or maybe is it worth giving some extra GBs of RAM to squeeze out last tiny percents of performance. And when adding more RAM starts making worse (in theory it is easy to achieve this especially if you measure your success in terms of latency and not throughput). However, any complex test is better done with more realistic workload. Here I want to check just the simple assumptions and dependencies.

Java GC, generational hypothesis and throughput

Thu, 30 Nov 2023 15:10:49 +0300

In recent years Java and Hotspot JVM in particular received several new garbage collectors. Most recently generational Z collector was introduced.

It is common knowledge that it is possible to achieve almost any results in artificial benchmarks, but nonetheless it may be interesting to create them anyway.

@State(Scope.Thread)
@Fork(value = 1, jvmArgs = {"-Xmx100m"})
@Warmup(iterations = 3, time = 1)
@Measurement(iterations = 5, time = 1)
public class GenerationsBenchmark {

 public static final int OBJECTS_1MB = 1024 * 1024 / 16;
 private ArrayList<Object> olds;

 @Param({"0", "20", "40", "60"})
 public int oldObjects;

 @Setup
 public void setup() {
 olds = new ArrayList<>(oldObjects * OBJECTS_1MB);
 for (int i = 0; i < oldObjects * OBJECTS_1MB; i++) {
 olds.add(new Object());
 }
 }

 @Benchmark
 @Fork(jvmArgsPrepend = {"-XX:+UseG1GC"})
 public void g1(Blackhole bh) {
 bench(bh);
 }

 private static void bench(Blackhole bh) {
 for (int i = 0; i < OBJECTS_1MB; i++) {
 bh.consume(new Object());
 }
 }

 public static void main(String[] args) throws RunnerException {
 Options opt = new OptionsBuilder().include(".*" + GenerationsBenchmark.class.getSimpleName() + ".*").build();

 new Runner(opt).run();
 }
}

First of all this benchmark is flawed that it is run with 1 fork of each configuration only. But results are quite stable and difference between configurations is so large, that we may save the time. Also, 100 MB of heap is not a common size for modern application, so this bench is truly artificial and micro. Some garbage collectors are not suited for such a small heap, while the others are optimized for this extreme too. So the comparison is not fair at all. Any judgments based on it should be applicable only in a similar very unusual situation.

Java map of maps of maps vs record keys

Thu, 05 Oct 2023 16:10:49 +0300

In the previous post I investigated using single map with struct keys verses using map of maps of maps in Go language. Here I want to run the same test for Java.

@State(Scope.Thread)
@BenchmarkMode(Mode.AverageTime)
@Fork(3)
@OutputTimeUnit(TimeUnit.MICROSECONDS)
@Warmup(iterations = 3, time = 1, timeUnit = TimeUnit.SECONDS)
@Measurement(iterations = 5, time = 1, timeUnit = TimeUnit.SECONDS)
public class MapMapMapBenchmark {

 @Benchmark
 public Map<Integer, Map<Integer, Map<Integer, String>>> mapMapMap() {
 var firsts = new HashMap<Integer, Map<Integer, Map<Integer, String>>>();
 test(((first, second, third, value) -> {
 var seconds = firsts.computeIfAbsent(first, unused -> new HashMap<>());
 var thirds = seconds.computeIfAbsent(second, unused -> new HashMap<>());
 thirds.put(third, value);
 }));
 return firsts;
 }

 @Benchmark
 public Map<Integer, Map<Integer, Map<Integer, String>>> mapMapMapExact() {
 var firsts = HashMap.<Integer, Map<Integer, Map<Integer, String>>>newHashMap(FIRSTS);
 test(((first, second, third, value) -> {
 var seconds = firsts.computeIfAbsent(first, unused -> HashMap.newHashMap(SECONDS));
 var thirds = seconds.computeIfAbsent(second, unused -> HashMap.newHashMap(THIRDS));
 thirds.put(third, value);
 }));
 return firsts;
 }

 record IntKey(int first, int second, int third) {
 }

 @Benchmark
 public Map<IntKey, String> singleMap() {
 var map = new HashMap<IntKey, String>();
 test((first, second, third, value) -> map.put(new IntKey(first, second, third), value));
 return map;
 }


 @Benchmark
 public Map<IntKey, String> singleMapExact() {
 var map = HashMap.<IntKey, String>newHashMap(FIRSTS * SECONDS * THIRDS);
 test((first, second, third, value) -> map.put(new IntKey(first, second, third), value));
 return map;
 }

 interface Tester {
 void accept(int first, int second, int third, String value);
 }

 //not static, not final to prevent constant folding (see JMHSample_10_ConstantFold)
 private int FIRSTS = 100;
 private int SECONDS = 10;
 private int THIRDS = 10;

 private void test(Tester tester) {
 for (int first = 0; first < FIRSTS; first++) {
 for (int second = 0; second < SECONDS; second++) {
 for (int third = 0; third < THIRDS; third++) {
 tester.accept(first, second, third, first + "-" + second + "-" + third);
 }
 }
 }
 }

 public static void main(String[] args) throws RunnerException {
 Options opt = new OptionsBuilder()
 .include(".*" + MapMapMapBenchmark.class.getSimpleName() + ".*")
 .addProfiler(GCProfiler.class)
 .build();

 new Runner(opt).run();
 }
}

Running it I got these results.

Cassandra lightweight transactions mixed with ordinary queries

Thu, 05 Oct 2023 15:10:49 +0300

Recently I’ve discovered that adding a simple “if exists” clause to query in Cassandra turns it into something completely different. Lightweight transaction is started that is not really compatible with running some data modification queries without such clause. I debugged this problem to the simple independent reproducer, so I’d like to share it.

@Testcontainers
class UpdateTest {

 @Container
 private static final CassandraContainer cassandra
 = (CassandraContainer) new CassandraContainer("cassandra:4.1.3").withExposedPorts(9042);

 private static CqlSession session;

 @BeforeAll
 static void setup() {
 session = CqlSession.builder()
 .addContactPoint(cassandra.getContactPoint())
 .withLocalDatacenter("datacenter1")
 .build();

 session.execute("drop keyspace if exists updatetests");
 session.execute("create keyspace updatetests with replication = {'class' : 'SimpleStrategy', 'replication_factor' : 1}");
 session.execute("use updatetests");
 session.execute("drop table if exists test");
 session.execute("create table test (a text, b int, primary key(a))");
 }

 @AfterAll
 static void tearDown() {
 session.close();
 }

 @Test
 void testLwtInsertAndLwtUpdate() {
 session.execute("insert into test(a,b) values(?,?) if not exists", "x", 1);
 session.execute("update test set b=? where a=? if exists", 2, "x");
 assertUpdate("x");
 }

 @Test
 void testInsertAndLwtUpdate() {
 session.execute("insert into test(a,b) values(?,?)", "y", 1);
 session.execute("update test set b=? where a=? if exists", 2, "y");
 assertUpdate("y");
 }

 @Test
 void testLwtInsertAndUpdate() {
 session.execute("insert into test(a,b) values(?,?) if not exists", "z", 1);
 session.execute("update test set b=? where a=?", 2, "z");
 assertUpdate("z");
 }
 private static void assertUpdate(String id) {
 var res = session.execute("select * from test where a=?", id);
 res.forEach(row -> assertEquals(2, row.getInt("b")));
 }
}

Running this sample only 2 out of 3 tests pass. testInsertAndLwtUpdate fails. What is even more interesting is that in my tests it reports lightweight transaction applied.

Go map of maps of maps vs complex keys

Thu, 05 Oct 2023 13:30:49 +0300

In other languages I’ve seen single map performing better than a map of maps or even map of maps of maps. I wanted to test which way is better in Go. So I wrote this test:

package main

import (
	"fmt"
	"testing"
)

type structID struct {
	first int
	second int
	third int
}

var Blackholemapmapmap map[int]map[int]map[int]string
var Blackholemapstruct map[structID]string

func BenchmarkName(b *testing.B) {
	b.Run("mapmapmap def maps", func(b *testing.B) {
		b.ReportAllocs()
		for i := 0; i < b.N; i++ {
			m := map[int]map[int]map[int]string{}
			test(func(first, second, third int, value string) {
				seconds := m[first]
				if seconds == nil {
					seconds = map[int]map[int]string{}
					m[first] = seconds
				}
				thirds := seconds[second]
				if thirds == nil {
					thirds = map[int]string{}
					seconds[second] = thirds
				}
				thirds[third] = value
			})
			Blackholemapmapmap = m
		}
	})
	b.Run("mapstruct def map", func(b *testing.B) {
		b.ReportAllocs()
		for i := 0; i < b.N; i++ {
			m := map[structID]string{}
			test(func(first, second, third int, value string) {
				m[structID{
					first: first,
					second: second,
					third: third,
				}] = value
			})
			Blackholemapstruct = m
		}
	})
	b.Run("mapmapmap exact maps", func(b *testing.B) {
		b.ReportAllocs()
		for i := 0; i < b.N; i++ {
			m := make(map[int]map[int]map[int]string, firstsCount)
			test(func(first, second, third int, value string) {
				seconds := m[first]
				if seconds == nil {
					seconds = make(map[int]map[int]string, secondsCount)
					m[first] = seconds
				}
				thirds := seconds[second]
				if thirds == nil {
					thirds = make(map[int]string, thirdsCount)
					seconds[second] = thirds
				}
				thirds[third] = value
			})
			Blackholemapmapmap = m
		}
	})
	b.Run("mapstruct exact map", func(b *testing.B) {
		b.ReportAllocs()
		for i := 0; i < b.N; i++ {
			m := make(map[structID]string, firstsCount*secondsCount*thirdsCount)
			test(func(first, second, third int, value string) {
				m[structID{
					first: first,
					second: second,
					third: third,
				}] = value
			})
			Blackholemapstruct = m
		}
	})
}

const (
	firstsCount = 100
	secondsCount = 10
	thirdsCount = 10
)

func test(add func(first, second, third int, value string)) {
	for first := 0; first < firstsCount; first++ {
		for second := 0; second < secondsCount; second++ {
			for third := 0; third < thirdsCount; third++ {
				add(first, second, third, fmt.Sprintf("%d-%d-%d", first, second, third))
			}
		}
	}
}

Running it I got these results.

Go channel(s) contention

Mon, 27 Mar 2023 20:22:39 +0300

In the previous post I’ve tried to measure overhead from highly contended access to sync.Pool on go. There was some measurable overhead, but it was mild. However in order for another goroutine to recieve some work to be done or/and to publish work results you may need go channel also. In this post I’ll try to measure the overhead of contended access to go channel.

So I’ve written the following code

Growing goroutine stacks

Fri, 24 Mar 2023 20:22:39 +0300

Go allows to run very lightweight goroutines and throw them away (for garbage collector) after they are no longer needed. You can execute any code inside them and this caode can allocate as much as needed. Both on stack and on heap. If goroutine stack is too small it grows. This grow is not free. Suppose you have a widely used function that needs to allocate a bit of memory on stack. Depending on whether there is enough of memory on stack or not it will take different amount of time, since in some cases it will include time to grow the stack. It happens that a really common function time.Now() is allocating. So this effect can be demoed with this function. Consider the following program.

Go sync.Pool contention

Fri, 24 Mar 2023 16:22:39 +0300

Recently my colleague suggested that contention on a single sync.Pool in Go may hit performance. I decided to reproduce this performance hit in a benchmark. This is what I was capable of writing.

import (
	"runtime"
	"strconv"
	"sync"
	"sync/atomic"
	"testing"
)

func workWithPool(pool *sync.Pool) {
	const size = 1000
	values := [size]int{}
	for i := 0; i < size; i++ {
		values[i] = pool.Get().(int)
	}
	for i := 0; i < size; i++ {
		pool.Put(values[i])
	}
}

func Benchmark_Pool(b *testing.B) {
	for _, parallelism := range []int{1, 2, 3, 4, 5, 10, 20, 50, 100} {
		b.Run("Parallelism "+strconv.Itoa(parallelism), func(b *testing.B) {

			b.Run("one pool", func(b *testing.B) {
				b.SetParallelism(parallelism)
				pool := &sync.Pool{
					New: func() any {
						return 42
					},
				}
				b.RunParallel(func(pb *testing.PB) {
					for pb.Next() {
						workWithPool(pool)
					}
				})
			})
			b.Run("many pools", func(b *testing.B) {
				b.SetParallelism(parallelism)
				numPools := runtime.GOMAXPROCS(0) * parallelism
				pools := make([]*sync.Pool, numPools)
				for i := 0; i < numPools; i++ {
					pools[i] = &sync.Pool{
						New: func() any {
							return 42
						},
					}
				}

				procs := uint32(0)
				b.RunParallel(func(pb *testing.PB) {
					poolidx := atomic.LoadUint32(&procs)
					for {
						if atomic.CompareAndSwapUint32(&procs, poolidx, poolidx+1) {
							break
						}
						poolidx = atomic.LoadUint32(&procs)
					}
					for pb.Next() {
						workWithPool(pools[poolidx])
					}
				})
			})
		})
	}
}

It seems that I need to comment a bit on the parallelism parameter. It stands for number of goroutines per each thread, while number of threads by default is equal to number of CPU cores. In my case I have 10 cores and thus 10 threads. Here are the results with some omissions and reformatting to simplify reading.

Java Loom Thread Fairness with Noisy Neighbour

Sun, 05 Jun 2022 13:30:49 +0300

Recently Gunnar Morling posted about Loom thread fairness. However despite the fact that difference between Loom and platform threads was clearly visible, Loom was even better. Total execution time was equal (or should be even better for Loom). But also there was some visible gradual progress.

However it is possible to get into situation where Loom virtual threads will behave worse than the platform threads. This situation is a pretty common pattern and it is called noisy neighbor problem. Nowadays we usually meet it microservices level, but the general idea is universal. So here is how it can be met inside a single JVM:

Java Cryptography Extension Providers Performance

Sat, 28 May 2022 20:44:46 +0300

The Java Cryptography Extensions exist for many years, They allow to use cryptography that is not bundled in the Java platform itself. I’ve met Bouncycastle crypto provider library many times in projects I worked on. But there are other third party crypto providers as well. At least two of them - Consrypt and Corretto (named the same way as Openjdk distribution by Amazon) declare high performance as their features. I decided to measure these claims and wrote some benchmarks. To not compare apples with oranges I needed some algorithms that were implemented in all different crypto providers that I tried to test. So the choice may seem a bit strange.

Go sync.Pool and gc

Tue, 01 Mar 2022 13:10:49 +0300

I’ve posted my brief experiments with sync.Pool in go. However, reading about it guides that it may be hard to use tool. And sometimes it can degrade for third-party reasons. It is easy to trap into old information regarding sync.Pool and gc interrogation. Since go 1.13 sync.Pool is not cleared completely on every gc. However, it is still affected by gc. Suppose the following code:

type Obj []int

func (o *Obj) Fill() {
	for i := 0; i < cap(*o); i++ {
		*o = append(*o, i)
	}
}

func (o Obj) Count() int {
	result := 0
	for i := 0; i < len(o); i++ {
		result += o[i]
	}
	return result
}

func NewObj() *Obj {
	o := make(Obj, 0, 100*1024)
	return &o
}

var objPool = &sync.Pool{
	New: func() interface{} {
		return NewObj()
	},
}

var ResultTrap int

func WorkWithPool() {
	o := objPool.Get().(*Obj)
	o.Fill()
	ResultTrap = o.Count()
	released := (*o)[:0]
	objPool.Put(&released)
}

And let’s use it this way:

Go sync.Pool

Fri, 25 Feb 2022 13:30:49 +0300

Recently I’ve seen sync.Pools in various Go libraries. Sometimes I wondered wether it is worth using at all. So I wrote some tests. Here are they:

package main

import (
	"crypto/sha256"
	"hash"
	"strings"
	"sync"
	"testing"
)

var shaPool = &sync.Pool{
	New: func() interface{} {
		return sha256.New()
	},
}

var HashTrap hash.Hash
var Source = []byte(strings.ToLower("asdfavzxcvsdfqawesfdz"))
var ResultTrap []byte

func Benchmark(b *testing.B) {
	b.Run("Pool", func(b *testing.B) {
		b.ReportAllocs()
		for i := 0; i < b.N; i++ {
			HashTrap = shaPool.Get().(hash.Hash)
			ResultTrap = HashTrap.Sum(Source)
			HashTrap.Reset()
			shaPool.Put(HashTrap)
		}
	})
	b.Run("NoPool", func(b *testing.B) {
		b.ReportAllocs()
		for i := 0; i < b.N; i++ {
			HashTrap = sha256.New()
			ResultTrap = HashTrap.Sum(Source)
		}
	})
	b.Run("Local", func(b *testing.B) {
		b.ReportAllocs()
		for i := 0; i < b.N; i++ {
			h := sha256.New()
			ResultTrap = h.Sum(Source)
		}
	})
}

And the results:

Go scheduler fairness

Fri, 02 Apr 2021 18:30:49 +0300

Recently in Jaeger I stumbled at an interesting trace - the work was done for 1 second, than 20 there was a gap for 20 seconds and after that work continued. My best guess currently is that goroutine slept for 20 seconds. To reproduce it I wrote the following:

package main

import (
	"context"
	"fmt"
	"runtime"
	"sync/atomic"
	"time"

	"github.com/couchbaselabs/ghistogram"
)

func main() {
	runtime.GOMAXPROCS(1)

	ctx, cancel := context.WithCancel(context.Background())
	workers := 300
	done := make([]chan bool, workers)
	for i := 0; i < workers; i++ {
		done[i] = make(chan bool)
	}

	times := ghistogram.NewHistogram(10, 5000, 0)
	minCount := int64(0)
	maxCount := int64(0)

	for i := 0; i < workers; i++ {
		doneChannel := done[i]
		count := int64(0)
		go func(ctx context.Context) {
			prev := time.Now()
			for {
				select {
				case <-ctx.Done():
					if count == 0 {
						println("Boom!")
					}
					for {
						savedMinCount := minCount
						if count < savedMinCount || savedMinCount == 0 {
							if atomic.CompareAndSwapInt64(&minCount, savedMinCount, count) {
								break
							}
						} else {
							break
						}
					}

					for {
						savedMaxCount := maxCount
						if count > savedMaxCount {
							if atomic.CompareAndSwapInt64(&maxCount, savedMaxCount, count) {
								break
							}
						} else {
							break
						}
					}
					close(doneChannel)
					return
				default:
					next := time.Now()
					dif := next.Sub(prev)
					times.Add(uint64(dif), 1)

					prev = next
					count++
				}
			}
		}(ctx)
	}

	time.Sleep(30 * time.Second)
	cancel()
	for i := 0; i < workers; i++ {
		<-done[i]
	}

	println("Times")
	for i, r := range times.Ranges {
		fmt.Printf("%s - %d\n", time.Duration(r), times.Counts[i])
	}
	fmt.Printf("min - %s\n", time.Duration(times.MinDataPoint))
	fmt.Printf("max - %s\n", time.Duration(times.MaxDataPoint))

	println("Counts")
	fmt.Printf("min - %d\n", int(minCount))
	fmt.Printf("max - %d\n", int(maxCount))
}

In this code I start 300 goroutines in 1 thread and try to iterate without any sleeps inside each goroutine. The only thing that is done in those loops - recording time of the iteration and storing it in histogram. Output is:

Moving work to another goroutine

Tue, 30 Mar 2021 12:25:42 +0300

Concurency in Go is super easy. Still to write it inside my very large work project I first needed to get my hands with it on really small and simple example. So I wrote one and it seems that this blog is a perfect place to store it.

package main

import (
	"time"
)

func main() {
	worker := make(chan int, 10)
	done := make(chan bool)

	go func() {
		for data := range worker {
			time.Sleep(50 * time.Millisecond)
			println(data)
		}
		close(done)
	}()

	for i := 1; i <= 20; i++ {
		worker <- i
	}

	close(worker)
	println("done settings tasks")
	<-done
}

The only thing that is too simplistic here is that we have only one worker goroutine. So let’s make some more - we need a separate bool channel to track finish.

Updating value in cache or simply invalidate it instead

Wed, 04 Nov 2020 10:51:48 +0300

Recently I worked on a process of updating many entities in DB. And since these entities are read really often, there was a Redis-based cache in front of DB. After updating the entity in DB it was updated in the cache. And to be safe the whole process was under Redsic-based lock. So:

Lock for the entity is taken in Redis
Entity is updated in DB
Entity is updated in Redis
Lost is released in Redis

Everything is good and data is consistent everywhere, but this process is too long. Lock acquire and release are the longest steps. But if we simply remove the locks data may become inconsistent between the DB and Redis. To craft such a sample is a too big task for a blog post, but a very similar concept can be recreated with any technology and at any level. In Java, there is Jcstress - an excellent tool to dive into concurrency problems. Let’s create a sample with it:

Profiling Java CPU and RAM with Mission Control

Sun, 01 Nov 2020 16:34:58 +0300

Some time ago I was a lecturer on Java courses and wanted to craft a sample for profiling Java applications. Currently, we have plenty of profilers, and more than that - many of them are completely free to use. One of them is Java Mission Control. It was a commercial offering with a limited free version, but now it is completely free and open-sourced. It was required to buy it to use in production back in the days and this alone should give a hint, that this profiler is so lightweight that it can be used in production. Now you can download builds of JMC from different vendors. I downloaded one from Azul.

Using map[]struct{} or map[]bool in Golang

Sun, 01 Nov 2020 13:35:52 +0300

In Golang you do not have set data structure. So you can use map instead. When you do not need values there are two options widely used - using empty struct{} or bool as value. struct{} should be more performant, while bool is more convinient. I decided to check the first promise myslef. Golang have benchamrks support in its standard library. SO I wrote this one:

package mapstruct

import (
	"math/rand"
	"testing"
)

func input() []int {
	slice := make([]int, 10_000)
	for i := 0; i < 10_000; i++ {
		slice[i] = rand.Intn(1_000)
	}
	return slice
}

func uniqueBool(input []int) []int {
	vals := map[int]bool{}
	for _, v := range input {
		vals[v] = true
	}
	result := make([]int, 0, len(vals))
	for v := range vals {
		result = append(result, v)
	}
	return result
}

func uniqueStruct(input []int) []int {
	vals := map[int]struct{}{}
	for _, v := range input {
		vals[v] = struct{}{}
	}
	result := make([]int, 0, len(vals))
	for v := range vals {
		result = append(result, v)
	}
	return result
}

func BenchmarkBool(b *testing.B) {
	for i := 0; i < b.N; i++ {
		uniqueBool(input())
	}
}

func BenchmarkStruct(b *testing.B) {
	for i := 0; i < b.N; i++ {
		uniqueStruct(input())
	}
}

And the result are quite variadic (Should developers write their own tooling to run benchamrks multiple times and compute averages, variances, etc?) Here are two results from different runs:

About

Sat, 31 Oct 2020 17:51:59 +0300

My name is Ivan Sopov. I am software developer working mostly with Java, Golang, various databases and similar things. You can find me on:

CTE Oracle vs Postgres

Sat, 15 Sep 2018 15:37:17 +0300

UPDATE(2021-11-08): Described here has changed in Postgres 12 (more recent versions have even better documentation on this).

Major part of my career to the moment passed working with databases and mostly with Oracle and Postgres. Several years ago I was really surprised by the difference in handling CTE (common table expressions) in these databases. From that time I spread this knowledge and recreated a sample for it several times. The last one was this week and I decided to store it somewhere - this mostly dead blog seems like a good place ;-) So here is a really simple case of using CTE on Oracle.

Benchmarking an interview question

Sat, 15 Aug 2015 23:22:39 +0300

Author benchmarks a super-simple method with a dumb implementation from his recent job interview. Results of this benchmark contradict with assumptions. The process of benchmarking produces some public benefit.

So, on a recent job interview it was suggested to write implementation for a really simple method

public int sum(int x, int y);

The main question for implementation is surely dealing with integer overflow (we were speaking about Java programming language). As it was left for me to decide what to do in case of overflow I suggested a standard way - throwing exception. In Java8 there is a standard method that does exactly this - Math.addExact. However I needed to re-implement it myself in order to use on older versions. To ease possible migration I used the same ArithmeticException that is used in the 8th version of standard library.

Learning Vaadin7 Review

Sat, 28 Dec 2013 17:42:46 +0300

Some time ago I was proposed to review the second edition of the “Learning Vaadin 7” (On Amazon) book. Shame on me for holding it for so long. Previously I have used Vaadin 6 and was really impressed by its features and simplicity of doing real desktop-like web-applications. But at the same time every now and then I was frustrated by some of the methods and mostly by the methods returning plain Objects. Fortunately now these type-safety problems are solved with the Vaadin 7. But what about the book? It seems as really good introduction to Vaadin for junior developers:

Github vs Bitbucket

Thu, 09 May 2013 09:30:30 +0300

Github totally owns the open-source movement today. And this situation impacts enterprise development. But as far as I can tell Bitbucket is much better for proprietary development in private repositories. Here are the arguments:

Option to prevent non-fast-forward pushes. It even allows to prevent non-fast-forward and still allowing removal of feature-branches! As far as I know, this may require a bit of configuration in standalone git repositories. ~~No such option on Github~~. UPDATE: Requires email to support@at@github.com but the response is fast - even in my not straightforward case.
Pricing model. Github offers unlimited collaborators form the start and additional private repos for additional money. Bitbucket offers unlimited private repos from the start and additional collaborators for additional money. I think the latter option is better - division of project in modules residing in separate repositories is the technical task. And technical decision person should have freedom to do the best. Technical staff usually does not make financial decisions. And extending the team is really that kind of decision that could not be made without considering financial side. So I think if choosing what side of the plan to make dependent on money - number of collaborators is better because without any Github or Bitbucket this side does depend on money. And number of repositories, if not made artificially dependent on money, does not depend on money at all.

So there are only two of them - ~~but the first I would consider as a show-stopper, and~~ the second really annoys leading to several unrelated projects in the single repository.

Use newer Java FTW!

Fri, 05 Apr 2013 09:44:46 +0300

One of my recent tasks was trying to convince client, that we should target the 7th version of the JVM, and not the 5th. I failed. Actually I don’t think that such task is possible unless there are no any real reasons in using older Java. Nevertheless here is the text, that I wrote.

What Java implementation are we going to use? The most used one is Oracle. Oracle Java5 reached its official end of life in Oct 2009 Oracle Java6 reached its official end of life in Feb 2013 (there will be no public updates anymore, nor any security updates)

Order of Enums in Hashmap

Fri, 10 Feb 2012 09:50:33 +0300

Recently I have found myself writing a complex sql-builder for two weeks already. Of cause I used TDD technique and this was much less like a pain comparing to the times when I didn’t wrote any tests. In some moment I decided that just operating with strings in some thousand or more lines of code is not very cool and started to write some OOD for the problem. In the resulting type hierarchy there were some enums. And I used these enums as keys in HashMap. Maybe some curious reader have already guessed what this post is about ;-) If not or if you just want some detailed illustration, read the rest:

ForkJoin factorial

Sun, 09 Oct 2011 10:05:54 +0300

Several months ago I wrote a multithreaded factorial method. It was very simple from the point of view of the underlying technology, but not so trivial from the point of view of synchronizing threads. It used simple start() and join() methods that are available since the Java 1.0 And than I thought that with all the power of java I can improve it. So i used the ThreadPoolExecutor - a piece of technology from the Java 5 and really improved - here is my post about multithreaded factorial using TreadPoolExecutor.

Multithreaded factorial with ThreadPoolExecutor

Fri, 22 Jul 2011 10:00:17 +0300

Update: the follow up post gives another version using newly introduced API.

In the previous post I have written a multithreaded factorial. It was not really good, because last one of the worker threads worked significantly longer than the first one. Fortunately we are living in 2011 and Java 1.5 was released so long time ago. The solution is simple - do not create 4 threads of the 4 virtual cores processors. Create really many threads and let the standard concurrent library execute them all. So ok, here is the code for this:

Multithreaded factorial

Sun, 03 Jul 2011 09:55:51 +0300

Update: the follow up post gives another version, that lacks the drawback of unequal load on processor cores.

There is a very good blog post about how to write the factorial method in Java (really it is not about righting factorial). However, even in this master peace;-) there is no multi-threaded factorial. So here it is. There are some problems with it. The most significant is that multiplying all the BigIntegers from 1 to 50000 is much less heavy job for the computer, than multiplying all BigIntegers from 50001 to 100000. So the the amount of work that is done by each worker-thread is not equal. But investigation on how this may be resolved in a common way is much harder problem, than just writing what I am publishing here. Maybe I’ll do it in the other post.

Not symmetric equals after override

Sat, 30 Apr 2011 10:23:54 +0300

There is really cool tool FindBugs and recently I found that it can discover (and discovered such a situation in my experience) broken symmetry for equals method when overriding it. So this is just a little example of this bug. (HashCode method is generated and I assume it to be all right).

public class BrokenEqualsTest {

 public static void main(String[] args) {
 Test1 test1 = new Test1();
 test1.field = "test";

 Test2 test2 = new Test2();
 ((Test1) test2).field = "test";
 test2.field2 = "test";

 System.out.println("test1 equals test2 - " + test1.equals(test2));
 System.out.println("test2 equals test1 - " + test2.equals(test1));
 }

 private static class Test1 {
 private String field;

 @Override
 public int hashCode() {
 final int prime = 31;
 int result = 1;
 result = prime * result + ((field == null) ? 0 : field.hashCode());
 return result;
 }

 @Override
 public boolean equals(Object obj) {
 if (obj instanceof Test1) {
 return field.equals(((Test1) obj).field);
 }
 return false;
 }
 }

 private static class Test2 extends Test1 {
 private String field2;

 @Override
 public int hashCode() {
 final int prime = 31;
 int result = super.hashCode();
 result = prime * result + ((field2 == null) ? 0 : field2.hashCode());
 return result;
 }

 @Override
 public boolean equals(Object obj) {
 if (obj instanceof Test2) { //here is the error reported by FindBugs
 return field2.equals(((Test2) obj).field2);
 }
 return false;
 }
 }
}

The output of running main of this class is:

How to change sid of Oracle Express (XE)

Sat, 30 Apr 2011 10:12:28 +0300

No way.

Collisions of hashcodes - much more real than an md5 collision

Sat, 26 Mar 2011 10:19:24 +0300

So okay, hashcode is int and so there are only 4294967296 distinct hashcodes. It is not a little number, but in modern systems there could be definitely much more objects.

In a very good book by Joshua Bloch I have read that there is a probability that two consequently created objects will have an equal system identity hashcode. Actually from the time of publishing that book major version of Java was released and the situation has improved dramatically. But still 232 is a very little number for objects. And still two consequently created objects may have an equal identity hashcode: