close
Skip to content

ci: add randomized matrix for better test coverage - #693

Merged
vlsi merged 3 commits into
apache:masterfrom
vlsi:ci_random_matrix
May 18, 2023
Merged

ci: add randomized matrix for better test coverage#693
vlsi merged 3 commits into
apache:masterfrom
vlsi:ci_random_matrix

Conversation

@vlsi

@vlsi vlsi commented Jan 15, 2022

Copy link
Copy Markdown
Collaborator

See https://github.com/vlsi/github-actions-random-matrix

Motivation and Context

This enables testing with different Java distributions, Java versions, Locales, Time Zones, etc.

Open issues so far:

  • Spock does not work in tr_TR locale: Unrecognized block label: given @ line 31, column 13. spockframework/spock#1414 (reproducer PR ci: add randomized test matrix for better test coverage spockframework/spock#1415)

  • Support ParallelExecutionConfigurationStrategy with Turkish locale junit-team/junit-framework#2890

  • Avoid relying on item.hashCode() for Map and Collection properties #5923

  • CSVDataSet might produce wrong results in SHARE_GROUP or SHARE_THREAD modes as it relies on System.identityHashCode uniqueness for ThreadGroup and Thread #5925

  • same hashcode jobs fail with. Fixed by moving HashTree to IdentityHashMap and moving Map<TestElement, .. to IdentityHashMap as well

    ERROR: unexpected output for BUG_62847.csv:
      - expected 247 bytes, /Users/runner/work/jmeter/jmeter/bin/testfiles/BUG_62847.csv
      + actual 86 bytes, /Users/runner/work/jmeter/jmeter/bin/BUG_62847.csv
    @@ -1,7 +1,2 @@
     label,responseCode,responseMessage,threadName,success
    -SuccessLoop,200,OK,TG 1-1,true
    -DS_after_loop,200,OK,TG 1-1,true
    -SuccessWhile,200,OK,TG 2-1,true
    -DS_after_while,200,OK,TG 2-1,true
    -Success_FEC,200,OK,TG 3-1,true
    -DS_after_fec,200,OK,TG 3-1,true
    +DS_after_fec,200,OK,TG 1-1,true
    ERROR: unexpected output for BUG_62847.xml:
      - expected 480 bytes, /Users/runner/work/jmeter/jmeter/bin/testfiles/BUG_62847.xml
      + actual 149 bytes, /Users/runner/work/jmeter/jmeter/bin/BUG_62847.xml
    @@ -1,10 +1,5 @@
     <?xml version="1.0" encoding="UTF-8"?>
     <testResults version="1.2">
    -<sample s="true" lb="SuccessLoop" rc="200" rm="OK" tn="TG 1-1"/>
    -<sample s="true" lb="DS_after_loop" rc="200" rm="OK" tn="TG 1-1"/>
    -<sample s="true" lb="SuccessWhile" rc="200" rm="OK" tn="TG 2-1"/>
    -<sample s="true" lb="DS_after_while" rc="200" rm="OK" tn="TG 2-1"/>
    -<sample s="true" lb="Success_FEC" rc="200" rm="OK" tn="TG 3-1"/>
    -<sample s="true" lb="DS_after_fec" rc="200" rm="OK" tn="TG 3-1"/>
    +<sample s="true" lb="DS_after_fec" rc="200" rm="OK" tn="TG 1-1"/>
  • Java 8 + same hashcode fails at :src:dist:processSiteXslt with VerifyError. I'm inclined to skip processSiteXslt when building with Java8 and same hashcode. We might retry if the bug is solved when xalan is updated.

    Error: Execution failed for task ':src:dist:processSiteXslt':
    : java.lang.VerifyError: (class: website_style, method: issue_separator signature: (Lcom/sun/org/apache/xalan/internal/xsltc/DOM;Lcom/sun/org/apache/xml/internal/dtm/DTMAxisIterator;Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;I)V) Illegal target of jump or branch
    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:109)
    Caused by: java.lang.VerifyError: (class: website_style, method: issue_separator signature: (Lcom/sun/org/apache/xalan/internal/xsltc/DOM;Lcom/sun/org/apache/xml/internal/dtm/DTMAxisIterator;Lcom/sun/org/apache/xml/internal/serializer/SerializationHandler;I)V) Illegal target of jump or branch
        at com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.getTransletInstance(TemplatesImpl.java:455)
        at com.sun.org.apache.xalan.internal.xsltc.trax.TemplatesImpl.newTransformer(TemplatesImpl.java:486)
    

@vlsi
vlsi force-pushed the ci_random_matrix branch 2 times, most recently from 40037be to bcdbacb Compare January 15, 2022 17:06
@codecov-commenter

codecov-commenter commented Jan 15, 2022

Copy link
Copy Markdown

Codecov Report

Merging #693 (f97d21a) into master (af7fc4d) will increase coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #693   +/-   ##
=========================================
  Coverage     55.59%   55.59%           
  Complexity    10336    10336           
=========================================
  Files          1059     1059           
  Lines         65045    65045           
  Branches       7399     7399           
=========================================
+ Hits          36160    36161    +1     
  Misses        26336    26336           
+ Partials       2549     2548    -1     
Impacted Files Coverage Δ
...n/java/org/apache/jmeter/reporters/Summariser.java 90.07% <0.00%> (-0.77%) ⬇️
...a/org/apache/jmeter/timers/PoissonRandomTimer.java 78.37% <0.00%> (+5.40%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update af7fc4d...f97d21a. Read the comment docs.

@vlsi
vlsi force-pushed the ci_random_matrix branch 4 times, most recently from b299a2f to c8db52a Compare January 15, 2022 17:53
@vlsi
vlsi force-pushed the ci_random_matrix branch from c8db52a to f97d21a Compare January 15, 2022 18:22
@vlsi
vlsi force-pushed the ci_random_matrix branch 2 times, most recently from 1bca384 to 8068fac Compare September 26, 2022 10:44
@vlsi
vlsi force-pushed the ci_random_matrix branch from 8068fac to bed24df Compare October 29, 2022 10:46
vlsi added a commit to vlsi/jmeter that referenced this pull request Oct 29, 2022
Use IdentityHashMap instead of HashMap when key is TestElement

See apache#693
@vlsi
vlsi force-pushed the ci_random_matrix branch from 28ebcb6 to 4157d37 Compare October 29, 2022 13:40
vlsi added a commit to vlsi/jmeter that referenced this pull request Oct 29, 2022
Use IdentityHashMap instead of HashMap when key is TestElement

See apache#693
@vlsi
vlsi force-pushed the ci_random_matrix branch 2 times, most recently from 5ce41d5 to 5e4ed16 Compare October 29, 2022 13:54
vlsi added a commit to vlsi/jmeter that referenced this pull request Oct 29, 2022
Use IdentityHashMap instead of HashMap when key is TestElement

See apache#693
@vlsi
vlsi force-pushed the ci_random_matrix branch from 5e4ed16 to 1ce3bb4 Compare October 29, 2022 14:02
vlsi added a commit to vlsi/jmeter that referenced this pull request Oct 29, 2022
Use IdentityHashMap instead of HashMap when key is TestElement

See apache#693
@vlsi
vlsi force-pushed the ci_random_matrix branch from 1ce3bb4 to e9b867e Compare October 29, 2022 14:08
vlsi added a commit to vlsi/jmeter that referenced this pull request Oct 29, 2022
Use IdentityHashMap instead of HashMap when key is TestElement

See apache#693
@vlsi
vlsi force-pushed the ci_random_matrix branch from e9b867e to db8984f Compare October 29, 2022 14:09
vlsi added a commit to vlsi/jmeter that referenced this pull request Oct 31, 2022
Use IdentityHashMap instead of HashMap when key is TestElement

See apache#693
@vlsi
vlsi force-pushed the ci_random_matrix branch from 7125112 to d5eb569 Compare October 31, 2022 10:07
vlsi added a commit to vlsi/jmeter that referenced this pull request Oct 31, 2022
Use IdentityHashMap instead of HashMap when key is TestElement

See apache#693
@vlsi

vlsi commented Nov 3, 2022

Copy link
Copy Markdown
Collaborator Author

@undera do you have any opinion on org.apache.jmeter.testelement.AbstractTestElement#equals ?

Currently AbstractTestElement#hashCode uses identityHashCode, so it was supposed to use reference equality when being a part of hashMap.
However, AbstractTestElement#equals used propMap.equal(other.propMap), so if someone accidentally puts TestElement in a HashMap, then they could get wrong results if hashCode values accidentally collide.

In this PR, I tried replacing the corresponding HashMap usages in JMeter with IdentityHashMap. It turns out it required many changes. The "good" part is that it keeps backward compatibility.
However, all the users would have to figure out and perform the same replacements.

There's another possibility: change AbstractTestElement#equals to final equals(Object o) { return this == o; }.
Then it would automatically support AbstractTestElement in HashMap and ConcurrentHashMap.
At the same time we could add AbstractTestElement#contentEquals for those who need to compare contents of the AbstractTestElement instead of their identity.

It looks like making AbstractTestElement#equals to compare object identity (this==that) would break backward compatibility (e.g. if someone used .equals to compare test element contents), however, it would automatically support TestElements in Set<...>, Map<....

WDYT?

@undera

undera commented Nov 3, 2022

Copy link
Copy Markdown

I never dug into this part of the code. From looking at the changes made, I'm a bit scared and confused by this change. This problem looks complex to understand, maybe you can draft a piece of code demonstrating the issue.

The base concept behind equals() and hashCode() should remain compliant to the original meaning from Java, to not cause strange effects. AbstractTestElement.equals() treats elements as equal based on their properties, including NAME, so it makes perfect sense, with exception of the TestElement's class mismatch possible (Assertion can match the Timer). Different TestElements would interpret the same set of properties differently, hence not equal. Let's consider code below.

    @Test
    public void testDemoProblem() throws Exception {
        AbstractTestElement o1 = new CompareAssertion();
        AbstractTestElement o2 = new ConstantTimer();
        Assert.assertEquals(o1.hashCode(), o2.hashCode()); // this one fails, makes sense
        Assert.assertEquals(true, o1.equals(o2)); // problem here?
    }

From the example above, we have a behavior that violates the idea of hashCode()+equal() from https://docs.oracle.com/javase/7/docs/api/java/lang/Object.html#hashCode()

Overall, I don't quite understand the issue and how real is it. I'd ask for a code piece to reproduce and illustrate the issue.

Maybe the problem is that makeProperty() is able to generate entries that are equal from different objects, but that's outside of this method's responsibility. I would state that it's on caller's responsibility to change the NAME or some other attribute of the resulting property, to make it distinguishable. Current caller just puts hashCode() into NAME, which is guaranteed to produce duplicates. Looks like the CollectionProperty would be the main place to look at, maybe the distinct name should be set inside it.

That's it. Not sure how my comments help :)

@vlsi

vlsi commented Nov 3, 2022

Copy link
Copy Markdown
Collaborator Author

Here's a problem:

    @Test
    public void testDemoProblem() throws Exception {
        AbstractTestElement o1 = new ConstantTimer();
        AbstractTestElement o2 = new ConstantTimer();
        Set<Timer> timers = new HashSet<>();
        timers.add(o1);
        timers.add(o2);
        Assert.assertEquals(2, timers.size(), "there should be 2 timers in the set");
    }

Pretty much the same issue happens in SearchByClass which is used to search ThreadGroup, ResultCollector, etc elements.
It is expected that SearchByClass would find all instances, however, if hash codes collide, then SearchByClass would produce fewer elements than present in the test plan.

Pretty much the same happens if you load .jmx file.

For instance, try doing the following:

  1. Run JMeter with -XX:+UnlockExperimentalVMOptions -XX:hashCode=2
$ export _JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:hashCode=2"
$ ./gradlew runGui
  1. Then open bin/testfiles/BUG_62847.jmx, and save it as bin/testfiles/BUG_62847_2.jmx

You'll see that JMeter discards two of three thread groups. In other words, I would expect that the file should be pretty much the same after re-save, however, the file is significantly different.

@undera

undera commented Nov 4, 2022

Copy link
Copy Markdown

I took the code snippet you suggested and it works fine for me (on master branch), just as expected, because the hashCode() is different.
IMO the example with HashSet behaves exactly how it should. It's a Set that is supposed to store only objects that are different by hashCode().

Can you arrange the code piece to demonstrate the issue with SearchByClass? I don't see in its source code any relation to equals() or hashCode(). The criteria for search there is searchClass.isAssignableFrom()

Also, if I open bin/testfiles/BUG_62847.jmx and re-save it with master branch code, it works fine. Let's understand the problem better and let's reproduce it on the level of unit tests.

For now, the only change I see needed is to make sure that test elements of different kind won't be matched.

@vlsi

vlsi commented Nov 4, 2022

Copy link
Copy Markdown
Collaborator Author

because the hashCode() is different.

Why are you so sure hashCode is always different?
There are at most 2**32 different hashCode values, so sometimes hashCode will collide.

If you launch OpenJDK with -XX:+UnlockExperimentalVMOptions -XX:hashCode=2, then the default Object#hashCode would return 1.

@vlsi

vlsi commented Nov 4, 2022

Copy link
Copy Markdown
Collaborator Author

Also, if I open bin/testfiles/BUG_62847.jmx and re-save it with master branch code

Did you perform export _JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:hashCode=2" before running "master branch code"?

@undera

undera commented Nov 4, 2022

Copy link
Copy Markdown

Did you perform export _JAVA_OPTIONS="-XX:+UnlockExperimentalVMOptions -XX:hashCode=2" before running "master branch code"?

No I did not. I don't understand what that option does and I stated above my full trust in standard approach around equals() and hashCode().

@undera

undera commented Nov 4, 2022

Copy link
Copy Markdown

Why are you so sure hashCode is always different? There are at most 2**32 different hashCode values, so sometimes hashCode will collide.

I went through your code piece under the debugger and I saw that the hashCode for two objects were different. The statement about 1/(2**32) event as sufficiently frequent does not sound to me. It's a very low probability and that's why Java relies on this result type of hashCode.

My point here is that I don't see any realistic evidence of a problem and struggling to understand it.

@vlsi

vlsi commented Nov 4, 2022

Copy link
Copy Markdown
Collaborator Author

The statement about 1/(2**32) event as sufficiently frequent does not sound to me

According to the birthday problem, the collision would happen with probability of 50% at 2**16 elements, which is not that uncommon.
Then, if something goes wrong, it would be extremely hard to identify: JMeter would skip certain samplers or configuration elements, so it is more like a silent data corruption rather than "crash once every 2**32 JMeter executions".

vlsi added a commit to vlsi/jmeter that referenced this pull request Nov 4, 2022
vlsi added a commit to vlsi/jmeter that referenced this pull request Nov 4, 2022
vlsi added a commit to vlsi/jmeter that referenced this pull request Nov 4, 2022
vlsi added a commit to vlsi/jmeter that referenced this pull request Nov 4, 2022
@undera

undera commented Nov 4, 2022

Copy link
Copy Markdown

I'm not sure where this discussion go to. Do we discuss Java's hashCode being unreliable?
I thought it's about some issue inside JMeter, though that issue does not show up in current master. I'm totally confused here.

@vlsi

vlsi commented Nov 6, 2022

Copy link
Copy Markdown
Collaborator Author

Sample failure in the current master:

public class SearchByClassTest {
    @Test
    public void test() {
        ListedHashTree tree = new ListedHashTree();
        int count = 100000;
        for (int i = 0; i < count; i++) {
            tree.add(new ThreadGroup());
        }
        SearchByClass<AbstractThreadGroup> searcher = new SearchByClass<>(AbstractThreadGroup.class);
        tree.traverse(searcher);
        Assertions.assertEquals(count, searcher.getSearchResults().size());
    }
}
org.opentest4j.AssertionFailedError: expected: <100000> but was: <99998>
	at app//org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
	at app//org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
	at app//org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
	at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
	at app//org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
	at app//org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:527)
	at app//org.apache.jorphan.collections.SearchByClassTest.test(SearchByClassTest.java:35)

@vlsi

vlsi commented Apr 29, 2023

Copy link
Copy Markdown
Collaborator Author

@FSchumacher , @pmouawad , @undera do you have any preference for the solution?

Just to remind: currently AbstractTestElement has mismatching equals and hashCode (see #693 (comment)) which breaks code like SearchByClass that might find fewer elements than it should (see sample in #693 (comment))

I have two possible solutions:

  1. Correct AbstractTestElement#hashCode to match AbstractTestElement#equals, so the compare contents (values in properties) rather that object identity. It would require many changes when TestElement is a key in HashMap. For instance, SearchByClass collects elements in a HashSet, and it would merge equal test elements, so we'll have to replace HashMap with IdentityHashMap in SearchByClass and similar places. See the current PR for the set of changes.

  2. Correct AbstractTestElement#equals to match AbstractTestElement#hashCode, so they use compare identity, and they always treat different objects unequal. It would automatically support cases when TestElement is placed in HashMap key, however, a new contentEquals method would be required to compare test element contents (e.g. when searching an argument. See ci: add randomized matrix for better test coverage defect, TestElement.contentEquals edition #5727 for the set of changes.

Unfortunately, both approaches will break backward compatibility one way or another. I do not think it is a severe breakage though, so I believe it is fine to have either of them in the upcoming 5.6.

I suggest going with approach 2, so we introduce contentEquals and contentHashCode methods. It seems a slightly smaller change overall, and it makes test elements safer to store in maps and sets. TestElement is mutable, so it would be slightly better if its hashCode and equals did not change as TestElement mutates.

However, I checked the sources of jmeter-plugins, jmeter-java-dsl, jmeter-maven-plugin and none of them seems to be impacted by this change (they do not store TestElement in HashMap, and they do not seem to use TestElement#equals)

WDYT?

@undera

undera commented Apr 29, 2023

Copy link
Copy Markdown

To me, the option #2 looks more logical.

@FSchumacher

Copy link
Copy Markdown
Contributor

If we change the behaviour, I think #2 would be best, too.

vlsi added a commit to vlsi/jmeter that referenced this pull request Apr 30, 2023
…Map instead of HashMap when key is TestElement

See apache#693
@vlsi
vlsi force-pushed the ci_random_matrix branch from c3ac696 to d7b470b Compare April 30, 2023 06:25
@vlsi

vlsi commented Apr 30, 2023

Copy link
Copy Markdown
Collaborator Author

I rebased "option 2", and now I'm puzzled.

It fails on test cases like assertEquals(new Header("accept", "*/*"), httpSampler.getHeaderManager().getHeader(0));

assertEquals(new Header("accept", "*/*"), httpSampler.getHeaderManager().getHeader(0));
assertEquals(new Header("X-XSRF-TOKEN", "1234"), httpSampler.getHeaderManager().getHeader(1));

The code that compares elements like if (file.equals(item)) { seems to be harder to spot.
Then, it is not that often that test elements are placed in sets. Well, JMeter itself does that, however, it is unlikely plugins will deal with Map<TestElement.

Then, we won't be able to make TestElement#equal final as it might break compatibility with some of the plugins that happened to override the method for their own reasons.

So it might be a safer choice to go for "option 1", and make TestElement#hashCode and TestElement#equals compare element contents. Then all the places that need element identity (e.g. SearchByClass that needs to report individual elements even if their contents is the same) should use IdentityHashMap or something like that.

vlsi added a commit to vlsi/jmeter that referenced this pull request May 1, 2023
…Map instead of HashMap when key is TestElement

See apache#693
@vlsi

vlsi commented May 17, 2023

Copy link
Copy Markdown
Collaborator Author

I'm going to merge this PR shortly unless there are objections.
An alternative #5727 is not dramatically smaller, and it breaks backward compatibility.

It looks like external users have much fewer usages for Map<TestElement, ...> than JMeter core, and TestElement#equals was probably used much more often for "comparing contents" in plugin code.

Unfortunately, the PR adds new usages of synchronizedMap, so we might need to replace them later. I believe they should not be dramatic

…Map instead of HashMap when key is TestElement

See apache#693
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants