ghc-compete, a repository of fingerprints, and continuous integration

[Herbert Valerio Riedel]

Hello Simon,

On 2013-10-15 at 14:45:05 +0200, Simon Marlow wrote:

[...]

> If we can get the tooling right I would probably have no objections to
> submodules.  Right now it's a bit of a pain though.
>
>  - extra steps in the workflow for modifying a library (git checkout
>    master, etc.)

Ok, this is the item that requires actual non-trivial scripting to
support; I'll need to think about how to best address this issue

How many extra steps in the workflow would be tolerable?

>  - possibility of losing local changes by git submodule update, if you
>    have made local changes in your copy of the library.
>    Perhaps what I want here is for "sync-all --rebase" to do "git
>    submodule update --rebase".

Fwiw, local uncommitted changes are warned about by recent Git versions:

,----
| $ git submodule update 
| error: Your local changes to the following files would be overwritten by checkout:
| 	prologue.txt
| Please, commit your changes or stash them before you can switch branches.
| Aborting
| Unable to checkout '6ad8c0d27bcff28c80684a29b57d7a8dbf00caca' in submodule path 'libraries/bytestring'
`----

As for committed changes, with "git config
submodule.<SUBMODULENAME>.update rebase" the --rebase flag can be made
the default setting; this could be configured up by 'sync-all' on
initial checkouts; so if 'git submodule update --rebase' is what you
want, this can be made the default mode easily even when not using
'sync-all'.

And finally, even if you happen to seemingly lose a commit due to a 'git
submodule update', there's still the 'git reflog' safeguard which keeps
a log of all HEAD updates and lets you recover recently "lost" commits.

>  - "sync-all new" doesn't work any more. I used to use this to see
>    what patches were unpushed in my tree relative to upstream.  I don't
>    mind what the command is, but there needs to be an easy way to do
>    this.

For this, Git itself already provides various commands:

1.) git submodule status

lists each submodule and reports if the submodule is "clean"; from the
man-page:

| Each SHA-1 will be prefixed with - if the submodule is not
| initialized, + if the currently checked out submodule commit does not
| match the SHA-1 found in the index of the containing repository and U
| if the submodule has merge conflicts.

or

2.) git submodule summary

This sounds a bit like what you want: Provides more verbose information
w.r.t. to the added/missing commits between the submodule's HEAD and
what is referenced by the main-repository (if you already committed the
submod-ref update in ghc.git, you could still generate a delta-summary
wrt the remote ghc.git by using "git submodule summary origin")

and finally

3.) git submodule foreach git status

The generic 'git submodule foreach' allows you to iterate
over all submodules, and call any command you can think of within the
submodule folder; in this case 'git status'

In fact, many current 'sync-all' operations which are simple iterations
can simply be replaced by an appropriate invocation of 'git submodule
foreach'.

>  - Difficulties with having a local mirror of the GHC repos.  Does
>    "sync-all -r <root> remove update-url" work with submodules now?
>    Does "./sync-all get" get the submodules from the same place as the
>    GHC repo?

btw, the command I assume you meant to write is probably

  sync-all -r <root> remote set-url

and there's been a commit some time ago for improving its behaviour on
submodules:

  http://git.haskell.org/ghc.git/commitdiff/0481e076f3cb4010894324cac71e947c6637805a

but it might not be perfect yet (I use the feature myself rather seldom,
so I don't know how well it works right now); however it's definitely
possible to make this work, and 'git submodule' does support rewriting
the urls between 'git submodule init' and 'git submodule update' via
per-submodule config variables, as well as after 'git submodule update'
has run via 'git remote set-url [--push]' -- so basically, I know what
needs to be done for this item in case it doesn't work already the way
it should.


So in summary, I think the first issue is the one that's a bit more
difficult to get right, as that's were the "tracking" semantics of Git
submodules become more apparent in the native git tooling.

Cheers,
  hvr