# HG changeset patch # User Jan Vrany # Date 1478045905 0 # Node ID 75b6eb7b781c10ba730e42be3be874021e35e993 # Parent 8d2d5dfe94d045bc3692b5ec3c68e24b792227f4 Added support for canonical, upstream and staging repositores. Each repository (forest) can now specify three repository URLS - (mandatory), "upstream: and "staging" repository (bothoptional). When a "staging" repository is configured, commits are first pulled from "staging" repository and then from "canonical" (assuming "staging" repository is local so this should avoid network trafic to canonical repositories). When an "upstream" repository is configured, changes are pulled from an "upstream" after pulling fron "staging" but before pulling from a canonical repository. This allows to define a hierarchy of repositories for staged development. This means, it allows for changes (commits) to go from one repository to another before eventually reaching a canonical repository from which official builds should be done. At each step commits should be verified and tested before they're pushed to upstream to avoid pushing broken code. diff -r 8d2d5dfe94d0 -r 75b6eb7b781c rakelib/hglib.rb --- a/rakelib/hglib.rb Sat Oct 29 23:54:12 2016 +0000 +++ b/rakelib/hglib.rb Wed Nov 02 00:18:25 2016 +0000 @@ -78,6 +78,9 @@ c_opts.reject! { | e | e.size == 0 } cmd = ['hg'] + g_opts + [command] + c_opts + args $LOGGER.debug("executing: #{cmd.shelljoin}") + if defined? RakeFileUtils then + puts cmd.shelljoin + end if block_given? then stdout, stderr, status = Open3.capture3(*cmd) case block.arity @@ -135,6 +138,13 @@ return HG::Repository.new(directory) end + # Initializes an empty repository in given directory. Returns an + # `HG::Repository` instance representing the created (empty) repository. + def self.init(directory) + HG::hg("init", directory) + return HG::Repository.new(directory) + end + # Like HG::hg, but passes --cwd @path def hg(command, *args, **options, &block) options[:cwd] = @path @@ -156,6 +166,18 @@ end end + # Return a hashmap with defined paths (alias => uri) + def paths() + return @config['paths'].clone + end + + # Set paths for given repository + def paths=(paths) + config = IniFile.new(:filename => self.hgrc()) + config['paths'] = paths + config.write() + end + def log(revset, template = "{node|short}\n") log = [] hg("log", rev: revset, template: template) do | status, out | diff -r 8d2d5dfe94d0 -r 75b6eb7b781c rakelib/rbspec.rb --- a/rakelib/rbspec.rb Sat Oct 29 23:54:12 2016 +0000 +++ b/rakelib/rbspec.rb Wed Nov 02 00:18:25 2016 +0000 @@ -494,8 +494,20 @@ class Repository < ConfigurationObject property :type, :class => Symbol - property :url - property :mirror + + # defines a canonical repossitory - this means, the one and only + # repository that contains "official" code. + property :canonical, :class => String + + # defines a so called upstream repository - this means the repository + # where the code from staging repository goes once verified. If no + # upstream repository is defined, canonical repository is taken + # instead + property :upstream, :class => String + + # defines staging repository + property :staging, :class => String + property :separator, :default => '.' @@ -518,6 +530,17 @@ return visitor.visit_repository(self) end + def upstream(value = '**token**') + if value != '**token**' then + self._set_property(:upstream, value) + else + if self.property_defined?(:upstream) + return self._get_property(:upstream) + else + return self.canonical + end + end + end end # class Repository diff -r 8d2d5dfe94d0 -r 75b6eb7b781c rakelib/scm.rb --- a/rakelib/scm.rb Sat Oct 29 23:54:12 2016 +0000 +++ b/rakelib/scm.rb Wed Nov 02 00:18:25 2016 +0000 @@ -28,7 +28,9 @@ end end - def self.update(type, repository, directory, *params) + def self.update(repository, directory, *params) + type = repository.type + url = repository.canonical self._check_type(type) if params.size() > 0 p = params.last @@ -38,7 +40,7 @@ root = p[:root] || BUILD_DIR wc = root / directory if (! File.exist? wc) - self.checkout(type, repository, directory, *params) + self.checkout(repository, directory, *params) return end case type @@ -49,22 +51,35 @@ end end - - def self._update_hg(wc, repository = nil, directory = nil, *params) - url = '' - if repository != nil and directory != nil then + def self._update_hg(wc, repository, directory = nil, *params) + if directory != nil then if params.size() > 0 p = params.last else p = {} end separator = p[:separator] || '.' - url = "#{repository}/#{directory.gsub('/', separator)}" + url = "#{repository.canonical}/#{directory.gsub('/', separator)}" end - repo = HG::Repository.new(wc) + hg = HG::Repository.new(wc) begin - repo.pull(url) - repo.update() + paths = hg.paths + if repository.staging then + if not paths.has_key? 'staging' + paths['staging'] = "#{repository.staging}/#{directory.gsub('/', separator)}" + hg.paths = paths + end + hg.pull('staging') + end + if not paths.has_key? 'canonical' + paths['canonical'] = "#{repository.canonical}/#{directory.gsub('/', separator)}" + hg.paths = paths + end + hg.pull('default') + if paths['default'] != paths['canonical'] then + hg.pull('canonical') + end + hg.update() rescue Exception => ex raise CheckoutException.new("HG: Cannot update #{wc}: #{ex.message}") end @@ -89,7 +104,9 @@ end end - def self.checkout(type, repository, directory, *params) + def self.checkout(repository, directory, *params) + type = repository.type + url = repository.canonical self._check_type(type) if params.size() > 0 p = params.last @@ -108,7 +125,7 @@ wc = root / directory if (File.exist? wc) - self.update(type, repository, directory, *params) + self.update(repository, directory, *params) return end @@ -129,7 +146,7 @@ end def self._checkout_svn(repository, directory, branch, root, *params) - url = "#{repository}/#{directory}/#{branch}" + url = "#{repository.canonical}/#{directory}/#{branch}" if not sh %W{svn --non-interactive --trust-server-cert co #{url} #{directory}}, cwd: root raise CheckoutException.new("SVN: Cannot checkout from #{url}") end @@ -143,18 +160,32 @@ end separator = p[:separator] || '.' - url = "#{repository}/#{directory.gsub('/', separator)}" + hg = HG::Repository.init(root / directory) + paths = { 'default' => "#{repository.upstream}/#{directory.gsub('/', separator)}", + 'canonical' => "#{repository.canonical}/#{directory.gsub('/', separator)}" } + if repository.staging then + paths['staging'] = "#{repository.staging}/#{directory.gsub('/', separator)}" + end + + hg.paths = paths + begin - repo = HG::Repository::clone(url, root / directory, noupdate: true) - repo.update(branch) + if repository.staging then + hg.pull('staging') + end + hg.pull('default') + if paths['default'] != paths['canonical'] then + hg.pull('canonical') + end + hg.update(branch) #rescue Exception => e # raise CheckoutException.new("HG: Cannot clone from #{url}: #{e.message}") end end def self._checkout_cvs(repository, directory, branch, root, *params) - if not sh %W{cvs -z 9 -d #{repository} co #{directory}}, cwd: root - raise CheckoutException.new("CVS: Cannot checkout #{directory}from #{repository}") + if not sh %W{cvs -z 9 -d #{repository.canonical} co #{directory}}, cwd: root + raise CheckoutException.new("CVS: Cannot checkout #{directory}from #{repository.url}") end end end # module Rake::Stx::SCM @@ -170,7 +201,7 @@ else params << {:separator => repo.separator} end - Rake::Stx::SCM.checkout(repo.type, repo.url, directory, *params) + Rake::Stx::SCM.checkout(repo, directory, *params) end def update(repo_name, directory, *params) @@ -179,22 +210,26 @@ if not repo then error("update(): No repository found (#{repo_name})") end - Rake::Stx::SCM.update(repo.type, repo.url, directory, *params) + Rake::Stx::SCM.update(repo, directory, *params) end -def cvs(repository, directory, *params) - Rake::Stx::SCM.checkout(:cvs, repository, directory, *params) +def cvs(url, directory, *params) + repo = Rake::Stx::Configuration::Repository.new(:type => :cvs, :url => url) + Rake::Stx::SCM.checkout(repo, directory, *params) end -def svn(repository, directory, *params) - Rake::Stx::SCM.checkout(:svn, repository, directory, *params) +def svn(url, directory, *params) + repo = Rake::Stx::Configuration::Repository.new(:type => :svn, :url => url) + Rake::Stx::SCM.checkout(repo, directory, *params) end -def hg(repository, directory, *params) - Rake::Stx::SCM.checkout(:hg, repository, directory, *params) +def hg(url, directory, *params) + repo = Rake::Stx::Configuration::Repository.new(:type => :hg, :url => url) + Rake::Stx::SCM.checkout(repo, directory, *params) end -def git(repository, directory, *params) - Rake::Stx::SCM.checkout(:git, repository, directory, *params) +def git(url, directory, *params) + repo = Rake::Stx::Configuration::Repository.new(:type => :git, :url => url) + Rake::Stx::SCM.checkout(repo, directory, *params) end diff -r 8d2d5dfe94d0 -r 75b6eb7b781c specs/repositories.rbspec --- a/specs/repositories.rbspec Sat Oct 29 23:54:12 2016 +0000 +++ b/specs/repositories.rbspec Wed Nov 02 00:18:25 2016 +0000 @@ -14,10 +14,12 @@ # This is the default if no REPOSITORYSET value is set. # # -# 2.*ci-swing*: for use on SWING CI [1]. Uses local mirrors. +# 2.*ci-swing*: for use on SWING CI [1]. Uses local staging repositories hosted +# on https://swing.fit.cvut.cz/hg in addition to canonical on hosted on +# BitBucket. # -# 3."ci-jv": for use on Jan Vrany's private CI. Uses local mirrors. -# +# 3."ci-jv": for use on Jan Vrany's private CI. Uses local staging repositories +# in addition to canonical on hosted on BitBucket. # REPOSITORYSET = (ENV['REPOSITORYSET'] || 'default') if not defined? REPOSITORYSET @@ -25,19 +27,19 @@ when 'default' repository :'exept:public' do type :cvs - url ":pserver:cvs@cvs.smalltalk-x.de:/cvs/stx" + canonical ":pserver:cvs@cvs.smalltalk-x.de:/cvs/stx" end repository :'bitbucket:janvrany' do type :hg - url "https://bitbucket.org/janvrany" + canonical "https://bitbucket.org/janvrany" separator '-' end if core_developer? repository :'swing:private:hg' do type :hg - url "ssh://192.168.12.2//hg" + canonical "ssh://192.168.12.2//hg" separator '.' end end @@ -45,19 +47,19 @@ when 'ci-swing' repository :'exept:public' do type :cvs - url ":ext:swing.fit.cvut.cz/var/local/cvs" + canonical ":ext:swing.fit.cvut.cz/var/local/cvs" end repository :'bitbucket:janvrany' do type :hg - url "https://bitbucket.org/janvrany" - mirror "ssh://swing.fit.cvut.cz//var/local/hg" + canonical "https://bitbucket.org/janvrany" + staging "ssh://swing.fit.cvut.cz//var/local/hg" separator '-' end repository :'swing:private:hg' do type :hg - url "ssh://192.168.12.2//hg" + canonical "ssh://192.168.12.2//hg" separator '.' end @@ -65,22 +67,19 @@ when 'ci-jv' repository :'exept:public' do type :cvs - url ":pserver:cvs@cvs.smalltalk-x.de:/cvs/stx" + canonical ":pserver:cvs@cvs.smalltalk-x.de:/cvs/stx" end repository :'bitbucket:janvrany' do type :hg - url "ssh://hg@192.168.0.250" + canonical "https://bitbucket.org/janvrany" + staging "ssh://hg@192.168.0.250" separator '-' end repository :'swing:private:hg' do type :hg - url "ssh://hg@192.168.0.250" + canonical "ssh://hg@192.168.0.250" separator '-' end - -end - -# -# +end \ No newline at end of file diff -r 8d2d5dfe94d0 -r 75b6eb7b781c specs/stx-jv.rbspec --- a/specs/stx-jv.rbspec Sat Oct 29 23:54:12 2016 +0000 +++ b/specs/stx-jv.rbspec Wed Nov 02 00:18:25 2016 +0000 @@ -204,48 +204,6 @@ # Smalltalk/X IDE application 'stx:projects/smalltalk', :repository => :'bitbucket:janvrany', :branch => 'jv' - if (ENV['USER'] == 'builder') || ( ENV['USER'] == 'vranyj1') || (ENV['USERNAME'] == 'builder') || ( ENV['USERNAME'] == 'vranyj1') then - repository :'mirror:swing' do - type :hg - url "https://swing.fit.cvut.cz/hg" - separator '-' - end - - stx_libjava_checkouter = - Proc.new do | pkg, build_dir | - repo = :'mirror:swing' - puts "Checking out #{pkg.name} from #{repo} (cache / staging repo)..." - checkout repo, pkg.directory, :branch => pkg.branch, :package => pkg, :separator => '-' - puts "Checking out #{pkg.name} from #{repo} (cache)...done" - - File.open(build_dir / 'stx' / 'libjava' / '.hg' / 'hgrc' , 'w') do | f | - f.puts "[paths]" - f.puts "default = https://bitbucket.org/janvrany/stx-libjava" - end - - repo = pkg.repository - puts "Checking out #{pkg.name} from #{repo}..." - update repo, pkg.directory, :branch => pkg.branch, :package => pkg, :separator => pkg._separator - puts "Checking out #{pkg.name} from #{repo}...done" - end - - stx_libjava_updater = - Proc.new do | pkg, build_dir | - repo = :'mirror:swing' - puts "Updating #{pkg.name} from #{repo} (cache / staging repo)..." - update repo, pkg.directory, :branch => pkg.branch, :package => pkg, :separator => '-' - puts "Checking out #{pkg.name} from #{repo} (cache)...done" - - repo = pkg.repository - puts "Checking out #{pkg.name} from #{repo}..." - update repo, pkg.directory, :branch => pkg.branch, :package => pkg, :separator => pkg._separator - puts "Checking out #{pkg.name} from #{repo}...done" - end - - - package "stx:libjava", :checkout => stx_libjava_checkouter, :update => stx_libjava_updater - end - package 'stx:libscm', :repository => :'bitbucket:janvrany' package 'stx:libscm/common', :repository => :'bitbucket:janvrany' package 'stx:libscm/mercurial', :repository => :'bitbucket:janvrany' @@ -258,6 +216,7 @@ package 'stx:goodies/smaCC',:repository => :'bitbucket:janvrany' package 'stx:goodies/regression',:repository => :'bitbucket:janvrany', :branch => 'jv', :link => false package "stx:goodies/builder", :repository => :'bitbucket:janvrany',:branch => 'jv', :link => false + package "stx:goodies/builder/reports" # Define a "default" test suite, i.e., a set of tests that are run when # user does `rake test`.