This is the final problem we need to fix before kotlin_1.3.31-1.deb can be fully rebuilt with kotlin_1.3.31-1.deb. I have tested this by copying kotlinx-coroutines-core to /usr/share/maven-repo/org/jetbrains/kotlinx/.
To ease the process of bootstrapping kotlin, it was felt that it would be best if we bundle kotlin and kotlinx-coroutines-core into a single package initially. @cdesai suggested that this can be done using Multiple Upstream Tarballs (MUT) approach that is supported in Debian. I will be working on doing this today learning about MUT support in uscan and gbp and combining the two builds. I would not mind someone more familiar with the approach beating me to it (even at the risk of some duplication of effort).
Imported v1.0.1 as component=coroutines from samyak-jn/kotlin@d6a5d6b8, now in master.
You'll need uscan (devscripts) from buster-backports or newer to properly handle the javascript style MUT versioning.
When using an downloaded version (built by upstream) of kotlin-coroutines-core.jar. The build still fail with above issue but when building coroutines component is commented out, the build fully succeeds. This is indicative that the this is due the version of kotlin compiler version we are using (1.3.30).
The kotlin version in the coroutines component (gradle.properties) indicates that preferred version of kotlin is 1.3.0.
Looking at the kotlin issue tracker we can find many similar crashes and reports. It is likely this is a bug with the compiler that was fixed in later version.
Based on this, I am trying to backport fixes to these issues. There seem to be multiple of them. I am currently trying a fix for issue KT-34708.
When I used Kotlin 1.3.31 from online maven repos, this error didn't occur. This means that there is a problem with how we build our kotlin.deb the bugs are not inherent in the compiler code. I plan to compare the downloaded files with the jars we build. They should more or less be identical. I am posting my download scripts here:
I have pused a commit to generate the correct version (1.3.31) in the jar file names.
I have posted a patch to utilize 1.3.31 as the upstream compiler version during stage1.
With the above two, I have reproduced the problem in stage2.
During stage 2 build, if I replace kotlin-compiler-embeddable.jar with upstream one, the problem got fixed. It stands to reason that there is an issue with the way are generating our kotlin-compiler-embeddable.jar (and not any of the other jars we are generating).
In the Ant build of Kotlin 0.8.x there is a parameter to generate the bootstrap compiler, it bundles all the dependencies, including the standard libraries, into the compiler jar. Maybe there is a similar mechanism in the Gradle build for Kotlin 1.3.x?
If I am reading the build system of the current version correctly, I don't see any option generate a single jar needed for bootstrap. The build system is, however, using the Gradle shadow plugin to combine several jars generated by subprojects into composite jars. I believe most of the jars generated by directories under the prepare/ directory are such. kotin-compiler-embeddable.jar is such a jar.
We may be able to combine the handful of jars needed for bootstrapping into a single jar by writing some gradle code using the shadow plugin. I wonder, however, if this will help us with debugging the issue.
During stage2 build, if we comment out the building of kotlinx-coroutines, the build of kotlin also fails with the following log:
...All input files are considered out-of-date for incremental task ':compiler:frontend.script:compileKotlin'. file or directory '/home/vagrant/kotlin/compiler/frontend.script/src/main/kotlin', not found file or directory '/home/vagrant/kotlin/compiler/frontend.script/src/main/java', not found file or directory '/home/vagrant/kotlin/compiler/frontend.script/src/main/kotlin', not found file or directory '/home/vagrant/kotlin/compiler/frontend.script/src/main/java', not found e: /home/vagrant/kotlin/compiler/frontend.script/src/org/jetbrains/kotlin/script/AsyncDependencyResolverWrapper.kt: (19, 8): Unresolved reference: kotlinx e: /home/vagrant/kotlin/compiler/frontend.script/src/org/jetbrains/kotlin/script/AsyncDependencyResolverWrapper.kt: (33, 15): Unresolved reference: runBlocking e: /home/vagrant/kotlin/compiler/frontend.script/src/org/jetbrains/kotlin/script/AsyncDependencyResolverWrapper.kt: (33, 38): Suspend function 'resolveAsync' should be called only from a coroutine or another suspend function :compiler:frontend.script:compileKotlin FAILED :compiler:frontend.script:compileKotlin (Thread[Task worker for ':' Thread 3,5,main]) completed. Took 1.493 secs. FAILURE: Build failed with an exception. ...
* Copyright 2010-2017 JetBrains s.r.o. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */package org.jetbrains.kotlin.scriptimport kotlinx.coroutines.runBlocking # kotlinx is missingimport kotlin.script.experimental.dependencies.DependenciesResolverimport kotlin.script.dependencies.Environmentimport kotlin.script.dependencies.ScriptContentsimport kotlin.script.experimental.dependencies.AsyncDependenciesResolver// wraps AsyncDependenciesResolver to provide implementation for synchronous DependenciesResolver::resolveclass AsyncDependencyResolverWrapper( override val delegate: AsyncDependenciesResolver): AsyncDependenciesResolver, DependencyResolverWrapper<AsyncDependenciesResolver> { override fun resolve( scriptContents: ScriptContents, environment: Environment ): DependenciesResolver.ResolveResult = runBlocking { delegate.resolveAsync(scriptContents, environment) } # as a result runBlocking is also missing and resolveAsync complains suspend override fun resolveAsync( scriptContents: ScriptContents, environment: Environment ): DependenciesResolver.ResolveResult = delegate.resolveAsync(scriptContents, environment)}
Here's the code it's referring to, since kotlinx-coroutines is missing, removing that just ends up breaking this code bit, leading to a compilation error. As a result the only problem is still the kotlinx coroutines thing that doesn't compile, nothing else I'm guessing.
kotlinx-coroutines.jar is being generated as part of stage1. However, that jar is empty. The empty jar is a consequence of skipping the :kotlinx-coroutines-core:transformAtomicfuClasses in debian/rules. This was done to fix a stage2 build failure while building :kotlinx-coroutines-core:. I am thinking of working around the problem in a different way to generated the jar file properly.
https://termbin.com/hjb1 as this log shows I think there was another reason for removing the rule, even stage one when being compiled by the bootstrap Kotlin fails :/
I'm recording this here because it may be useful later, but I think it was either a dead end or I didn't do it properly. Building without the --exclude-task fails with ClassNotFoundException: kotlinx.coroutines.UndispatchedCoroutine which exists in the kotlinx-coroutines-core-common project. However that jar was missing from libkotlin-bootstrap-java which only has kotlinx-coroutines-core, so I downloaded it from MavenCentral and repacked the .deb but got the same error.
I added a patch to fix an issue in backend code generation:
d/patches: Add patch to fix crash when generating coroutinesCloses: #9.When generating JVM byte code for a coroutine, code is inserted to help stop thefunction and continue is later. As part of this a variable with name'$continuation' is also added. This variable is also added to the local variabletable (LVT). The entry in LVT contains that scope of the variable represented asa starting label (inclusive) and ending label (exclusive).Before adding a starting label, the starting label is first created. At the timeof creation of the label and at the time of assing the starting label node tothe variable in LVT, different LabelNode are created. This works in a flaky waybecause of a reference (.info) in the label in LVT. It somehow does not workwhen compiled in Debian due change in ASM Java library or for other reasons.This patch fixes the problem by using the same LabelNode at the location of thelabel and in LVT. This problem was fixed in later versions of Kotlin due toractoring.This patch is not needed upstream due to refactoring done in commitd2a80e793825c16506a235a6dc61a6a052835110.Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
After this, I was able to fully rebuild kotlin (tested three stages so far) from our built kotlin.deb.