Version 56 (modified by 6 years ago) ( diff ) | ,
---|
Building Smalltalk/X jv-branch from sources
This text describes how to compile Smalltalk/X jv-branch from sources. The same process and tools is used on SWING CI server to build Smalltalk/X jv-branch packages.
In a Nutshell
A quick guide for the impatient...
- Make sure you have set up a build environment as instructed
- Open a shell (on Linux) or command prompt (on Windows) and execute:
hg clone https://bitbucket.org/janvrany/stx-goodies-builder-rake stx-jv cd stx-jv rake
- Go out for a short or long walk, depending on your internet connection and CPU power.
Once rake
finishes (without an error, indeed) everthing should be compiled. You can then run Smalltalk/X by:
projects/smalltalk/smalltalk
or on Windows:
projects\smalltalk\smalltalk.bat
If compilation fails, please read the rest of this document, especially section on compiling, updating and hacking. If it is still failing, feel free to ask.
Have Fun!
Background
A little background information on what buildscripts actually do...
Building Smalltalk/X jv-branch is a little complex and requires number of steps:
- First, you need to fetch all the sources from various repositories. As of now, from Jan Vrany's BitBucket and from eXept's public CVS. All sources has to be put into proper directory structure.
- Second, as source of stc and librun is not publicly available, pre-build binaries have to be downloaded. As of now, they're downloaded from (former) SWING Research Group CI Server.
- Various patches and tweaks has to be applied to sources in order to compile.
- Finally compile all packages in dependency order
Build scripts utomate all necessary steps so it all happens automagically - ideally it all should be as easy as typing
rake
Not always it is like that, but we aim to make it just like this :-)
Setting up a Build Environment
To build a Smalltalk/X you need to have a number of tools installed and set up, such as C compiler toolchain. These tools are not needed to run Smalltalk/X once compiled.
The rest of this section gives you a detailed information on how to set up your system. Generally speaking, you need following:
- A GCC compiler - 4.7.x, 5.x or 6.x
- GNU Make
- Mercurial and CVS clients
- Ruby 2.1 or newer
- rake
- JDK 7
- Apache Ant
Windows
Tested on Window 7 Professional, 64bit.
- Install Mercurial from https://www.mercurial-scm.org/. Make sure the path to hg.exe is in your
PATH
. - Install Ruby 2.2.x from http://rubyinstaller.org/downloads/. Make sure Ruby interpreter is in your
PATH
(by checking corresponding checkbox when installing it) - the best way to install ruby is to use ruby manager like uru (https://bitbucket.org/jonforums/uru)
- Install MSYS2 from https://msys2.github.io/. Make sure to install it to
C:\MSYS64
, otherwise Smalltalk/X build scripts won't find it!. Once installer finishes, open a command prompt and execute following commands:c:\msys64\usr\bin\pacman.exe -Sy pacman c:\msys64\usr\bin\pacman.exe -Syu c:\msys64\usr\bin\pacman.exe -Su
Note: during pacman -Syu
you maybe asked to close command prompt window. Don't forget to close it otherwise you will have to start from beginning.
Note: when you want to start use MSYS2 shell use msys2_shell.cmd
(installer installs incorrectly .bat
files).
- Install GNU Make and CVS client. To do so, execute following in a command prompt:
c:\msys64\usr\bin\pacman.exe -S make cvs openssh ssh-pageant-git zip unzip
- Add
c:\msys64\usr\bin
to yourPATH
:setx PATH "%PATH%;c:\msys64\usr\bin"
Close the command prompt an open it again before continuing''' This is important so the newPATH
become effective. - Install GCC toolchain. For building 64bit Smalltalk/X, install:
c:\msys64\usr\bin\pacman.exe -S mingw-w64-x86_64-gcc mingw-w64-x86_64-SDL
For 32bit builds, install:c:\msys64\usr\bin\pacman.exe -S mingw-w64-i686-gcc mingw-w64-i686-SDL
You may install both if you want to build both, 64bit and 32bit versions. - Install Java JDK 7 or newer. Though Oracle Java would do, we recommend using Zulu from http://www.azul.com/downloads/zulu/zulu-windows/ az Zulu 7 is recommended for use with STX:LIBJAVA on Windows. Make sure
java.exe
andjavac.exe
are in yourPATH
. To do so, you may use following commands:setx PATH "%PATH%;C:\Program Files\Zulu\zulu-7\bin" setx JAVA_HOME "C:\Program Files\Zulu\zulu-7"
Close the command prompt an open it again before continuing''' This is important so the newPATH
become effective. - Install Apache Ant - https://wiki.apache.org/ant/AntOnWindows. Make sure
ant
command is in yourPATH
.
This should be everything you need to compile Smalltalk/X jv-branch as described above
Linux
Following guides are for building 64bit Smalltalk/X on 64bit Linux (or 32bit Smalltalk/X on 32bit Linux). It is possible - of course - to build ac 32bit Smalltalk/X on 64bit Linux. In that case you need to install 32bit libraries and headers. How to do that is left as an exercise to dear reader/. Hint: on DEB-based distros you need to install pretty much same packages with :i386 suffix, on RPM-based with .i686 plus something like gcc-multilib.
Debian (and other derivatives such as Ubuntu or Mint)
Tested on Debian Stretch, both 64bit and 32bit, standard installations.
Install following packages:
sudo aptitude install ruby mercurial cvs build-essential bison flex libc6-dev gcc-multilib libx11-dev default-jdk ant libglib2.0-dev libwebkitgtk-dev libsoup2.4-dev libxext-dev unixodbc-dev libgl1-mesa-dev libfl-dev
This should be everything you need to compile Smalltalk/X jv-branch as described above
Building 32bit Smalltalk/X on 64bit system
To build 32bit Smalltalk/X on 64bit system, you need to install 32bit versions of some libraries in addition to above:
dpkg --add-architecture i386 apt-get update apt-get install libx11-dev:i386 libxext-dev:i386 libxft2:i386 libxinerama-dev:i386 libodbc1:i386 odbcinst1debian2:i386 libgl1-mesa-dev:i386 libfl-dev:i386
CentOS 7
Tested on CentOS 7.2 (Minimal). Thanks to Patrik Svestka for this guide!
First couple steps only install X environment. If you already have it installed or don't want it, you may skip them.
- First install XServer (with xfce)
sudo yum groupinstall "Development Tools" "X Window system" xfce Fonts systemctl set-default graphical.target
- Optional: For the system to have "nice" fonts
sudo yum install xorg-x11-fonts-Type1 xorg-x11-fonts-misc
Following is required to compile Smalltalk/X jv-branch
- Install rvm (management for ruby - system ruby is not a good idea):
rvm requirements rvm install ruby-2.3 rvm docs generate-ri # documentation
- Install java (open-jdk 8)
sudo yum install java
- unixODBC needed to start StX (installs handy ODBC connector)
sudo yum install unixODBC
- Versioning systems (for getting the sources from)
sudo yum install mercurial cvs
- Devel packages needed for compilation
sudo yum install glib2 libX11-devel libXext-devel qtwebkit libsoup libXft-devel libXinerama-devel ant bison flex
This should be everything you need to compile Smalltalk/X jv-branch as described above
- Optional: Install XSLT and DocBook Stylesheets. This is only required if you want to create deployment artifacts, i.e., run
rake install
orrake artifacts
sudo yum install libxslt-devel docbook5-style-xsl docbook-style-xsl
Since paths on CentOS differ from those on Debian, workaround this by creating symlinks:cd /usr/share/xml/ mkdir -p docbook/stylesheet/ cd /usr/share/xml/docbook/stylesheet ln -s /usr/share/sgml/docbook/xsl-stylesheets-1.78.1/ docbook-xsl
Fedora 23
Tested on Fedora 23 Workstation, 64bit, standard installation. Thanks to Martin Kobetic and Patrik Svestka for hints.
- Install following packages:
sudo dnf install gcc java ant ruby rake mercurial cvs bison flex unixODBC glib2 libX11-devel libXext-devel qtwebkit libsoup libXft-devel libXinerama-devel ant
This should be everything you need to compile Smalltalk/X jv-branch as described above
openSUSE Tumbleweed
Tested on openSUSE Tumbleweed on June 29th, 2018. Thanks to Patrik Svestka for testing and this guide.
- Install ruby via rvm:
rvm install ruby
- Needed packages:
sudo zypper install mercurial fontconfig fontconfig-devel bison flex unixODBC unixODBC-devel glib2 libX11-devel libXext-devel qtwebkit libsoup libXft-devel libXinerama-devel ant
- Deal with java:
The stx:libjava
currently supports only java-openjdk 1.7 so you have to install it on your system to be able to work with it. This java is no longer available in the default repositories. You need to add the Java bootstrap
repository in order to install Java openjdk 1.7.0
.
First you need to add a Java bootstrap
repository:
Java bootstrap repostitory via command (see the link for other mirrors):
sudo zypper ar -f http://mirror.karneval.cz/pub/linux/opensuse/repositories/Java:/bootstrap/openSUSE_Tumbleweed/Java:bootstrap.repo
Then you will be asked to verify the key fingerprint:
New repository or package signing key received: Repository: Attic project for packages that are needed to bootstrap java without java (openSUSE_Tumbleweed) Key Name: Java OBS Project <Java@build.opensuse.org> Key Fingerprint: 97119219 72E27C87 BBC1BA89 E38C29BC 4276E0B9 Key Created: Mon 18 Jun 2018 03:42:22 PM CEST Key Expires: Wed 26 Aug 2020 03:42:22 PM CEST Rpm Name: gpg-pubkey-4276e0b9-5b27b6be
After you have verified the fingerprint you can press t
to trust it and you will add the needed repository.
Then check all the available java 1.7.0 openjdk versions with:
xxx@localhost:~/build/java7> sudo zypper search java-1_7 Loading repository data... Reading installed packages... S | Name | Summary | Type ---+---------------------------------------+------------------------------------------------------------------+----------- i+ | java-1_7_0-openjdk | SUSE's implementation of the OpenJDK 7 runtime environment | package | java-1_7_0-openjdk | SUSE's implementation of the OpenJDK 7 runtime environment | srcpackage | java-1_7_0-openjdk-accessibility | Accessibility connector for OpenJDK 7 | package | java-1_7_0-openjdk-debuginfo | Debug information for package java-1_7_0-openjdk | package | java-1_7_0-openjdk-debugsource | Debug sources for package java-1_7_0-openjdk | package | java-1_7_0-openjdk-demo | OpenJDK 7 Demos | package | java-1_7_0-openjdk-demo-debuginfo | Debug information for package java-1_7_0-openjdk-demo | package i+ | java-1_7_0-openjdk-devel | SUSE's implementation of the OpenJDK 7 Development Environment | package | java-1_7_0-openjdk-devel-debuginfo | Debug information for package java-1_7_0-openjdk-devel | package i+ | java-1_7_0-openjdk-headless | OpenJDK 7 runtime environment without X, audio and video support | package | java-1_7_0-openjdk-headless-debuginfo | Debug information for package java-1_7_0-openjdk-headless | package | java-1_7_0-openjdk-javadoc | OpenJDK 7 API Documentation | package | java-1_7_0-openjdk-src | OpenJDK 7 Source Bundle | package
You can then install the Java openjdk 1.7.0 with
sudo zypper install java-1_7_0-openjdk java-1_7_0-openjdk-devel java-1_7_0-openjdk-headless
Note: When installing these rpm(s) you will probably see a warning like this one:
java-1_7_0-openjdk-devel-1.7.0.181-2.5.x86_64.rpm: Header V3 DSA/SHA1 Signature, key ID 4276e0b9: NOKEY V3 DSA/SHA1 Signature, key ID 4276e0b9: NOKEY
Which means that you don't have the signature key with which the packages were signed installed. That is why it is important to check the Key Fingerprint when adding the repository. (If you can find the key you can add it too).
To check currently installed and activated java:
sudo /usr/sbin/update-alternatives --config java
Note: update-alternatives - maintain symbolic links determining default commands
You could get the following output:
There are 4 choices for the alternative java (providing /usr/bin/java). |
Selection | Path | Priority | Status |
0 | /usr/lib64/jvm/jre-1.8.0-oracle/bin/java | 2700 | auto mode |
* 1 | /usr/lib64/jvm/jre-1.7.0-openjdk/bin/java | 1705 | manual mode |
2 | /usr/lib64/jvm/jre-1.8.0-openjdk/bin/java | 1805 | manual mode |
3 | /usr/lib64/jvm/jre-1.8.0-oracle/bin/java | 2700 | manual mode |
4 | /usr/lib64/jvm/jre-11-openjdk/bin/java | 2105 | manual mode |
Press <enter> to keep the current choice[*], or type selection number: |
When you activate it you will see a message:
Press <enter> to keep the current choice[*], or type selection number: 1 update-alternatives: using /usr/lib64/jvm/jre-1.7.0-openjdk/bin/java to provide /usr/bin/java (java) in manual mode
Note: An excerpt from *update-alternatives* man pages to explain the above table.
Each link group is, at any given time, in one of two modes: automatic or manual. When a group is in automatic mode, the alternatives system will automatically decide, as packages are installed and removed, whether and how to update the links. In manual mode, the alternatives system will retain the choice of the administrator and avoid changing the links (except when something is broken).
Link groups are in automatic mode when they are first introduced to the system. If the system administrator makes changes to the system's automatic settings, this will be noticed the next time update-alternatives is run on the changed link's group, and the group will automatically be switched to manual mode.
Each alternative has a priority associated with it. When a link group is in automatic mode, the alternatives pointed to by members of the group will be those which have the highest priority.
When using the --config option, update-alternatives will list all of the choices for the link group of which given name is the master alternative name. The current choice is marked with a ‘*’. You will then be prompted for your choice regarding this link group. Depending on the choice made, the link group might no longer be in auto mode. You will need to use the --auto option in order to return to the automatic mode (or you can rerun --config and select the entry marked as automatic).
After all these steps you can check if the correct java version is now active:
xxx@localhost:~>java -version
java version "1.7.0_181" OpenJDK Runtime Environment (IcedTea 2.6.14) (build 1.7.0_181 suse-2.5-x86_64) OpenJDK 64-Bit Server VM (build 24.181-b01, mixed mode)
xxx@localhost:~> env | grep JAVA
JAVA_ROOT=/usr/lib64/jvm/java-1.7.0 JAVA_HOME=/usr/lib64/jvm/java-1.7.0 JAVA_BINDIR=/usr/lib64/jvm/java-1.7.0/bin
If you see similar results as above then you have installed your Java OpenJDK 1.7.0
successfully.
FreeBSD
Note: FreeBSD port is still under development. As of now (2018-08-01) it runs, the IDE opens but there are still some problems with Java. It does not pass standard test suite, not yet. Stay tuned.
Install following packages using pkg
:
sudo pkg install bash cvs mercurial ruby rubygem-rake flex bison openjdk-7.111.01 unixODBC apache-ant-1.9.7 pkgconf
Compiling, recompiling, updating and hacking
The whole compilation is driven by rake, a Ruby build tool. All daily tasks are implemented as rake targets and therefore can be invoked by simply running
rake <target>
in a shell (command prompt).
Remember, build scripts are also versioned and evolve (slowly, but they do), so you'd better update them regularly.
Task: checkout fresh sources and compile
To fetch all the source code use checkout
target, i.e,
rake checkout
This will check all necessary sources from corresponding repositories into build
subdirectory.
To compile, use - surprise - compiler
target.
Task: update previously checked out sources to latest revisions
rake update
will update every repository to the most recent revision (aka HEAD or master). Then use
rake compile
to recompile. Following a custom, this would recompile only what changed (and what depends on changed sources).
To compile only one package, for example, smallsense
you have to do it the following way:
rake stx:goodies/smallsense
Task: run compiled Smalltalk/X
To run compiled Smalltalk/X, go to build/stx/projects/smalltalk
directory and run smalltalk
(smalltalk.bat
on Windows):
cd build/stx/projects/smalltalk ./smalltalk -I --quick
(-I
means no snapshot restart, --quick
means no search for autoloaded classes)
Task: run tests to make sure everything is fine
Once compiled (or hacked) you may want to make sure that everything is fine by running tests. As you may have guessed, use rake
to run test
task:
rake test
Beware that it may take a quite some time to run all the tests. Make sure you're not running on battery
as it may go flat rather quickly. Ideally, all tests should pass. Once all run, a short summary is printed. You may find detailed XML report in reports
directory. If not all of them pass, please report an issue!
Task: TODO
To be written...
Building Smalltalk/X on Jenkins CI
Work in progress. This section may be incomplete needs to be polished.
Setting up Jenkins Server
Smalltalk/X pipeline script uses a number of API which Pipeline sandbox plugin does not allow. To allow all of them at once, edit $JENKINS_HOME/scriptApproval.xml
and add following:
<approvedSignatures> <string>method com.cloudbees.plugins.credentials.common.IdCredentials getId</string> <string>method com.cloudbees.plugins.credentials.common.StandardCredentials getDescription</string> <string>method hudson.plugins.mercurial.MercurialSCM getBranch</string> <string>method hudson.plugins.mercurial.MercurialSCM getCredentialsId</string> <string>method hudson.plugins.mercurial.MercurialSCM getSource</string> <string>method java.lang.Class getName</string> <string>method java.lang.Object hashCode</string> <string>method java.lang.Class isInstance java.lang.Object</string> <string>method java.util.Collection remove java.lang.Object</string> <string>method java.util.Collection toArray</string> <string>new java.util.ArrayList java.util.Collection</string> <string>new java.util.HashMap</string> <string>new java.util.HashMap java.util.Map</string> <string>staticMethod com.cloudbees.plugins.credentials.CredentialsProvider lookupCredentials java.lang.Class</string> <string>staticMethod java.lang.System identityHashCode java.lang.Object</string> <string>staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods contains java.lang.Object[] java.lang.Object</string> <string>staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods plus java.util.List java.lang.Iterable</string> <string>staticMethod org.codehaus.groovy.runtime.DefaultGroovyMethods putAt java.lang.Object java.lang.String java.lang.Object</string> <string>method java.util.Calendar add int int</string> <string>method java.util.Calendar getTimeInMillis</string> <string>method java.util.Calendar set int int</string> </approvedSignatures>
Setting up Build Slaves
In addition to setup described above, additional packages / setup is needed when running a full CI pipeline on a build slave.
Debian (and alike)
Install following extra packages:
aptitude install xsltproc docbook-xsl
Windows
Install following extra packages:
pacman -Su libxslt docbook-xsl
IMPORTANT: make sure you run c:\msys64\msys2.exe
as the jenkins user. This creates MSYS2 home directory for the user - otherwise SSH won't be able to store known_hosts
file there and checkout would fail.
See MSYS ssh error: Could not create directory '/home/<username>/.ssh'
NOTE: If using Windows 10, you may want to enable long paths by default. To so so, set registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled
to 1
(as 32-bit DWORD).
Troubleshooting
When things go wrong, check this troubleshooting checklist for common problems (and solutions).