Raised on behalf of @kuksenko .

In multiple places, it uses Random to generate random numbers. It should be ThreadLocalRandom since they runs in a multi-threaded environment.

Heapothesys should have a mode to allow modeling of application behavior. This includes object size variability, variable allocation rates, peak and off-peak allocation, and potentially other ideas.

In SimpleRunner.java:80 cast 1000 to 1000L to prevent loss of precision.

For a given JVM garbage collector and Extremem workload configuration, it is desirable to automate the exploration of what range of JVM configurations (e.g. how many cores, how much heap memory) is required to sustain compliance with a particular range of client response time jitter.

In particular, suppose we want to assure 99.9% of response times to be less than 10ms. How large does the heap need to be and how many cores must be dedicated to GC?

The pom.xml file has a few typos (Heapothesys vs heapothesys), and incorrect URLs. Fixup

Raised on behalf of @kuksenko .

For measuring elapsed time, we should use System.nanoTime() here, as David Holmes mentioned a long time ago.

Hi team - is the heapothesys project dead?
Has anyone used it with JDK 21?

The following errow was reported by CI testing for a new pull request.

2020-08-05T18:58:09.6886330Z [INFO] Running com.amazon.corretto.benchmark.hyperalloc.ObjectStoreTest
2020-08-05T18:58:10.9727619Z [ERROR] Tests run: 9, Failures: 0, Errors: 1, Skipped: 1, Time elapsed: 1.257 s <<< FAILURE! - in com.amazon.corretto.benchmark.hyperalloc.ObjectStoreTest
2020-08-05T18:58:10.9743146Z [ERROR] ReshuffleTest Time elapsed: 0.09 s <<< ERROR!
2020-08-05T18:58:10.9744135Z java.util.ConcurrentModificationException
2020-08-05T18:58:10.9746866Z at com.amazon.corretto.benchmark.hyperalloc.ObjectStoreTest.lambda$ReshuffleTest$1(ObjectStoreTest.java:123)
2020-08-05T18:58:10.9749358Z at com.amazon.corretto.benchmark.hyperalloc.ObjectStoreTest.ReshuffleTest(ObjectStoreTest.java:123)

The configuration options of Extremem are designed to allow Extremem to mimic the memory workload behavior of broad varieties of real-world applications and services. These workloads are typically characterized in terms of:

1. allocation rate trends
2. live memory usage trends
3. object longevity demographics (e.g. 70% of objects die within 50 ms, additional 20% within 10 s, other 10% within 5 minutes)

If customers have good understandings of their services in terms of memory allocation behaviors, it would be desirable that the documentation of Extremem configuration choices would guide them in selection of the configuration parameters that would cause Extremem to approximately match the memory behavior of their service.

In the absence of this documentation, customers are required to select experimental configuration choices, run Extremem with GC logging enabled, compare the GC log output to the comparable GC log output for their service, and then repeat the experiment with adjustments to the experimental configuration choices, iterating until the observed behavior is sufficiently close to the workload they are trying to model.

The goal is to simplify capture, sharing, and repetition of workload configurations between different runs.

Revise tests to avoid this dependency as jdk.nashorn is no longer bundled with most current OpenJDK releases.

Setup Actions to build heapothesys packages

Replace all usage of printStackTrace() with a proper logging mechanism.

Real applications also share the available processing power with JVM threads (GC in this case). Update Heapothesys to support adding variable application CPU utilization.

Raised on behalf of @kuksenko .

Since it knows the number of threads required, it should used FixedThreadPool in SimpleRunner.java. The CacheThreadPool doesn't allow to control threads properly, may exhaust resources.

As more developers contribute to the implementation of Extremem, it is desirable to establish a suite of test programs and a framework for applying a regression test suite on the Extremem workload.

1. Steps to reproduce

• Run heapothesys with -m value, where value >= 1024

2. The queueSize calculated for allocation workers in SimpleRunner.java->createTasks method becomes negative due to integer overflow. For eg. -m = 1024, the intermediate value in queueSize calculation is (2^31)/(config.getMaxObjectSize() + config.getMinObjectSize())/config.getNumOfThreads(). And 2^31 > Integer.MAX_VALUE.

I'm happy to update the POM file to explicitly version plugins etc so that heapothesys can have repeatable builds. The downside is a larger pom.

Scripts to automate the running of Extremem and especially the assimilation of output reports would be very helpful. This would facilitate use of Extremem to support performance and latency regression testing.

Without these scripts, running an Extremem workload involves considerable human intervention to search the output and assess the significance of the generated reports.

As currently implemented, Extremem is best suited to modeling of application services that are running in steady state mode (allocating temporary objects at roughly the same pace they are discarding objects).

Extremem as currently impelmetned does not model workloads that depend on finalization or reference processing.

It also does not effectively model phase changes in which large numbers of newly allocated objects will have relatively long lifetimes (e.g. application startup and initialization) or in which large numbers of previously promoted objects are becoming garbage.

New workloads and configuration options for Extremem are required.

The version and build number should be written to standard output with the reporting of other configuration information.

Adding this information to the output helps make comparison of results between different runs more meaningful and helps assure repeatability of test results. (In theory, it may not be meaningful to compare results from version N with results from version N+1.)

JDK 14 errors at:

java.lang.ExceptionInInitializerError
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method)
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstanceWithCaller (Constructor.java:500)
...
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 1
at org.codehaus.plexus.archiver.zip.AbstractZipArchiver. (AbstractZipArchiver.java:116)
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method)
at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstanceWithCaller (Constructor.java:500)
at java.lang.reflect.Constructor.newInstance (Constructor.java:481)
at com.google.inject.internal.DefaultConstructionProxyFactory$ReflectiveProxy.newInstance (DefaultConstructionProxyFactory.java:126)
at com.google.inject.internal.ConstructorInjector.provision (ConstructorInjector.java:114)
at com.google.inject.internal.ConstructorInjector.access$000 (ConstructorInjector.java:32)
at com.google.inject.internal.ConstructorInjector$1.call (ConstructorInjector.java:98)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:112)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:127)
at com.google.inject.internal.ProvisionListenerStackCallback.provision (ProvisionListenerStackCallback.java:66)
at com.google.inject.internal.ConstructorInjector.construct (ConstructorInjector.java:93)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get (ConstructorBindingImpl.java:306)
at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
at com.google.inject.internal.InjectorImpl.getInstance (InjectorImpl.java:1086)
at org.eclipse.sisu.space.AbstractDeferredClass.get (AbstractDeferredClass.java:48)
at com.google.inject.internal.ProviderInternalFactory.provision (ProviderInternalFactory.java:85)
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision (InternalFactoryToInitializableAdapter.java:57)
at com.google.inject.internal.ProviderInternalFactory$1.call (ProviderInternalFactory.java:66)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:112)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:127)
at com.google.inject.internal.ProvisionListenerStackCallback.provision (ProvisionListenerStackCallback.java:66)
at com.google.inject.internal.ProviderInternalFactory.circularGet (ProviderInternalFactory.java:61)
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get (InternalFactoryToInitializableAdapter.java:47)
at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
at org.eclipse.sisu.inject.Guice4$1.get (Guice4.java:162)
at org.eclipse.sisu.inject.LazyBeanEntry.getValue (LazyBeanEntry.java:81)
at org.eclipse.sisu.plexus.LazyPlexusBean.getValue (LazyPlexusBean.java:51)
at org.eclipse.sisu.plexus.PlexusRequirements$RequirementProvider.get (PlexusRequirements.java:250)
at org.eclipse.sisu.plexus.ProvidedPropertyBinding.injectProperty (ProvidedPropertyBinding.java:48)
at org.eclipse.sisu.bean.BeanInjector.injectMembers (BeanInjector.java:52)
at com.google.inject.internal.MembersInjectorImpl.injectMembers (MembersInjectorImpl.java:160)
at com.google.inject.internal.ConstructorInjector.provision (ConstructorInjector.java:124)
at com.google.inject.internal.ConstructorInjector.access$000 (ConstructorInjector.java:32)
at com.google.inject.internal.ConstructorInjector$1.call (ConstructorInjector.java:98)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:112)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:127)
at com.google.inject.internal.ProvisionListenerStackCallback.provision (ProvisionListenerStackCallback.java:66)
at com.google.inject.internal.ConstructorInjector.construct (ConstructorInjector.java:93)
at com.google.inject.internal.ConstructorBindingImpl$Factory.get (ConstructorBindingImpl.java:306)
at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
at com.google.inject.internal.InjectorImpl.getInstance (InjectorImpl.java:1086)
at org.eclipse.sisu.space.AbstractDeferredClass.get (AbstractDeferredClass.java:48)
at com.google.inject.internal.ProviderInternalFactory.provision (ProviderInternalFactory.java:85)
at com.google.inject.internal.InternalFactoryToInitializableAdapter.provision (InternalFactoryToInitializableAdapter.java:57)
at com.google.inject.internal.ProviderInternalFactory$1.call (ProviderInternalFactory.java:66)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:112)
at com.google.inject.internal.ProvisionListenerStackCallback$Provision.provision (ProvisionListenerStackCallback.java:127)
at com.google.inject.internal.ProvisionListenerStackCallback.provision (ProvisionListenerStackCallback.java:66)
at com.google.inject.internal.ProviderInternalFactory.circularGet (ProviderInternalFactory.java:61)
at com.google.inject.internal.InternalFactoryToInitializableAdapter.get (InternalFactoryToInitializableAdapter.java:47)
at com.google.inject.internal.InjectorImpl$1.get (InjectorImpl.java:1050)
at org.eclipse.sisu.inject.Guice4$1.get (Guice4.java:162)
at org.eclipse.sisu.inject.LazyBeanEntry.getValue (LazyBeanEntry.java:81)
at org.eclipse.sisu.plexus.LazyPlexusBean.getValue (LazyPlexusBean.java:51)
at org.codehaus.plexus.DefaultPlexusContainer.lookup (DefaultPlexusContainer.java:263)
at org.codehaus.plexus.DefaultPlexusContainer.lookup (DefaultPlexusContainer.java:255)
at org.apache.maven.plugin.internal.DefaultMavenPluginManager.getConfiguredMojo (DefaultMavenPluginManager.java:520)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:124)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:564)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)

When I run the benchmark the process does not exit when finished, seems due to the threads keeping the JVM alive, adding executor.shutdown(); to SimpleRunner seems to solve this

Part of the computational workload shouldered by Extremem is to track the total allocations performed by each thread along with tracking when previously allocated objects should become eligible for garbage collection. These tallies are reported to standard output.

Nearly all objects allocated by Extremem are eventually supposed to be discarded. At certain critical execution points, the total number and flavors of certain object allocations should equal the total number and flavors of matching object discards. That these numbers do not currently match demonstrates that there are errors in the current calculations of these memory allocation and deallocation tallies.

We need to correct the tallies.

Hi All,

This is not an issue but a request for clarification on how to use this benchmark. I used the following flags to run (copied from the README page:
-Xmx65536m -Xms6553 -XX:+UnlockExperimentalVMOptions -XX:+UseLargePages -XX:+UseHugeTLBFS -XX:+UseTransparentHugePages -XX:+AlwaysPreTouch -XX:-UseBiasedLocking -Xloggc:.XX_gc.log '-javaagent:/home/ubuntu/jHiccup-2.0.10/jHiccup.jar=-a -d 0 -i 1000 -l .XX.hlog' -jar /home/ubuntu/heapothesys/target/Heapothesys-1.0.jar -a 16384 -h 32768 -d 600 -m 128 -c false -t 64 -n 64 -x 32768 -l .XX.csv

The process does not stop even after the duration time 10 mins. Also may I know how to analyze the output? I get a csv file and two log files. May I know how to visualize them (if there is a way provided by the project)?

Thanks

1. Steps to reproduce:

• Run heapothesys with program args as -a 200 -h 1024 -d 60 -m 50 -s 100 -c true and VM args as -XX:+UnlockExperimentalVMOptions -Xms1g -Xmx1g -XX:G1NewSizePercent=60 -Xlog:gc*=info,gc+age=trace -XX:SurvivorRatio=3 -XX:TargetSurvivorRatio=95

2. Ideally, after promotion of 100MB to old gen, we should consistently see 50MB in the survivor space. But, we see the entire 150MB in the old gen.
3. In the attached gc log file, check the logs after line 290 (gc(15)).
   OS info - Windows 10 v1909
   JDK - OpenJDK 64-Bit Server VM (build 14.0.1+7, mixed mode, sharing)
   bad_case.txt