[m-dev.] New release?

[Paul Bone]

Simple question, difficult answer..

Is now a good time for a new Mercury release?

Last time Julien and I spoke about this, quite a while ago, I recall that we
wanted to upgrade Boehm GC first. Now that that is done is there any other
planned changes that we'd like to wait for?

I've noticed that Zoltan has been doing a lot of refactoring. Is it best to
wait until this settles down and has been tested in ROTDs for a little
while? Or is that what the release candidates/betas are for?

Cheers.

[Matthias Güdemann]

Post by Paul Bone
Is now a good time for a new Mercury release?

I am in favor of having a new release, as the last one was a long time
ago. I can offer testing release candidates on my collection of Mercury
programs.

Best regards,
Matthias

[Zoltan Somogyi]

Post by Paul Bone
Last time Julien and I spoke about this, quite a while ago, I recall that we
wanted to upgrade Boehm GC first. Now that that is done is there any other
planned changes that we'd like to wait for?
I've noticed that Zoltan has been doing a lot of refactoring. Is it best to
wait until this settles down and has been tested in ROTDs for a little
while? Or is that what the release candidates/betas are for?

At the moment, the statuses of insts and modes are recorded twice;
once with the old status system, and once with a new. All operations
on those statuses are done on both, the results compared, and an abort
generated if they disagree. Since this has been going for a month now,
I think it is safe to say that there aren't any problems with the new system,
and we can switch over to it without the crutch of checking against the old
system. I think we should do so before the release.

However, I am in the middle of changing over to a similar system for
the statuses of types, and any diff that implemented the previous paragraph
would cause conflicts for me. I would prefer to wait a week or two before
starting the release process, for two things: (a) me to finish the diff for type
statuses, and (b) time to wait for any problems with that diff to turn up.

There is also another bug we need to fix before the release, one I am working
on now. It seems that under some circumstances, the building of either the
.opt or the .trans_opt file can put invalid data into the .d file of the relevant
module, which requires an "mmake depend" before you can create the .c file
of that module. This bug was probably added with my changes to intermod.m,
but it showed itself only when working on the library directory, since that is
the only part of the Mercury system we turn on intermodule optimization for.

Zoltan.

[Julien Fischer]

Hi Paul,

Post by Paul Bone
Simple question, difficult answer..
Is now a good time for a new Mercury release?

It's certainly time we had another one; the GC upgrade was the main block.

Post by Paul Bone
Last time Julien and I spoke about this, quite a while ago, I recall that we
wanted to upgrade Boehm GC first. Now that that is done is there any other
planned changes that we'd like to wait for?

Before we branch we need to finalise the break-up of the standard library's
'term' module, as per the following thread
<http://www.mercurylang.org/list-archives/reviews/2015-February/017741.html>.
In particular, the type term.context/0 needs to be moved to its own module.

Bug #357, "Parallel conjunction broken on OS X" is a showstopper. The
actual situation is worse than described in that bug report, _none_ of the
.par grades currently work on OS X. (They worked in 14.01.)

In addition, the following bugs really ought to be resolved before
any release:

* Bug #278: Compiler crash from bad state variable use.
* Bug #373: string.to_float imprecisely specified.

(There's almost certainly some other bugs that we would ideally address as
well.)

There's some more standard library issues that aren't currently in the bug
database, for example float.round_to_int/1 and friends have inconsistent
behaviours across different backends when their argument won't fit in an int.
(I'll add an entry to Mantis for this.)

There was a list of things we wanted to do to address the "too many grades"
problem. Some of these have been done (e.g. hiding user facing documentation
for experimental grade components), others haven't yet (e.g. merging the 'tr'
and 'trseg' grade components and renaming the grade component options). See
the following thread:
<http://www.mercurylang.org/list-archives/reviews/2014-September/017297.html>

Post by Paul Bone
I've noticed that Zoltan has been doing a lot of refactoring. Is it best to
wait until this settles down and has been tested in ROTDs for a little
while?

Answered elsewhere in this thread.

Post by Paul Bone
Or is that what the release candidates/betas are for?

Ideally, they are not for testing large scale restructures such as the ones
Zoltan has been working on. We will need to let things stablise first.

Julien.

[Paul Bone]

Post by Julien Fischer
Hi Paul,

Post by Paul Bone
Simple question, difficult answer..
Is now a good time for a new Mercury release?

It's certainly time we had another one; the GC upgrade was the main block.

Post by Paul Bone
Last time Julien and I spoke about this, quite a while ago, I recall that we
wanted to upgrade Boehm GC first. Now that that is done is there any other
planned changes that we'd like to wait for?

Before we branch we need to finalise the break-up of the standard library's
'term' module, as per the following thread
<http://www.mercurylang.org/list-archives/reviews/2015-February/017741.html>.
In particular, the type term.context/0 needs to be moved to its own module.
Bug #357, "Parallel conjunction broken on OS X" is a showstopper. The
actual situation is worse than described in that bug report, _none_ of the
.par grades currently work on OS X. (They worked in 14.01.)

I've also been having trouble with parallel conjunction lately. I'm not
sure what's changed as I haven't looked at this for a while. Although very
few people use this feature, I'd still prefer to avoid regressions from
release-to-release.

These types of bugs can probably be fixed on a release branch and merged
back into master. That said it still makes sense to wait before branching
for Zoltan's change, to avoid making that more difficult than it needs to
be.

Post by Julien Fischer
In addition, the following bugs really ought to be resolved before
* Bug #278: Compiler crash from bad state variable use.
* Bug #373: string.to_float imprecisely specified.
(There's almost certainly some other bugs that we would ideally address as
well.)
There's some more standard library issues that aren't currently in the bug
database, for example float.round_to_int/1 and friends have inconsistent
behaviours across different backends when their argument won't fit in an int.
(I'll add an entry to Mantis for this.)

Hrm, I'd like to try making a "TODO list" of bugs to fix for the release.
I suggest tagging bugs in Mantis with the release name then using the
roadmap feature as a TODO list. I've added #357 as a test:

https://www.mercurylang.org/bugs/roadmap_page.php

Post by Julien Fischer
There was a list of things we wanted to do to address the "too many grades"
problem. Some of these have been done (e.g. hiding user facing documentation
for experimental grade components), others haven't yet (e.g. merging the 'tr'
and 'trseg' grade components and renaming the grade component options). See
<http://www.mercurylang.org/list-archives/reviews/2014-September/017297.html>

When this came up recently Micheal (despite how he raised the issue) made a
good suggestion that we should document not just what the grade components
mean (which mostly exists in the user guide) but which combinations make
sense / are supported / are well tested. For example it's not obvious to a
new user that hlc.gc.stseg does not make sense.

This was one of the reasons I setup the wiki. There were a bunch of reasons
but this was the one that actually got me to do something. It has has very
few contributions but zero spam, so I think that there's still potential
there. Anyway, my point is that some of the "too many grades" solution is
orthogonal to making a release.

Post by Julien Fischer

Post by Paul Bone
I've noticed that Zoltan has been doing a lot of refactoring. Is it best to
wait until this settles down and has been tested in ROTDs for a little
while?

Answered elsewhere in this thread.

Post by Paul Bone
Or is that what the release candidates/betas are for?

Ideally, they are not for testing large scale restructures such as the ones
Zoltan has been working on. We will need to let things stablise first.

Agreed, not the large scale stuff.

[Julien Fischer]

Post by Paul Bone

Post by Julien Fischer
Hi Paul,

Post by Paul Bone
Simple question, difficult answer..
Is now a good time for a new Mercury release?

It's certainly time we had another one; the GC upgrade was the main block.

Post by Paul Bone
Last time Julien and I spoke about this, quite a while ago, I recall that we
wanted to upgrade Boehm GC first. Now that that is done is there any other
planned changes that we'd like to wait for?

Before we branch we need to finalise the break-up of the standard library's
'term' module, as per the following thread
<http://www.mercurylang.org/list-archives/reviews/2015-February/017741.html>.
In particular, the type term.context/0 needs to be moved to its own module.
Bug #357, "Parallel conjunction broken on OS X" is a showstopper. The
actual situation is worse than described in that bug report, _none_ of the
.par grades currently work on OS X. (They worked in 14.01.)

I've also been having trouble with parallel conjunction lately. I'm not
sure what's changed as I haven't looked at this for a while. Although very
few people use this feature, I'd still prefer to avoid regressions from
release-to-release.

When I say the situation is worse, I mean that hlc.par.gc *also* does
not currently work on OS X (i.e. you can't use threads in the C grades
on OS X at the moment.)

Post by Paul Bone
These types of bugs can probably be fixed on a release branch and merged
back into master. That said it still makes sense to wait before branching
for Zoltan's change, to avoid making that more difficult than it needs to
be.

Post by Julien Fischer
In addition, the following bugs really ought to be resolved before
* Bug #278: Compiler crash from bad state variable use.
* Bug #373: string.to_float imprecisely specified.
(There's almost certainly some other bugs that we would ideally address as
well.)
There's some more standard library issues that aren't currently in the bug
database, for example float.round_to_int/1 and friends have inconsistent
behaviours across different backends when their argument won't fit in an int.
(I'll add an entry to Mantis for this.)

Hrm, I'd like to try making a "TODO list" of bugs to fix for the release.
I suggest tagging bugs in Mantis with the release name then using the
https://www.mercurylang.org/bugs/roadmap_page.php

It seems a trifle optimistic to assume that it's going to be 15.11 ;-)

Post by Paul Bone

Post by Julien Fischer
There was a list of things we wanted to do to address the "too many grades"
problem. Some of these have been done (e.g. hiding user facing documentation
for experimental grade components), others haven't yet (e.g. merging the 'tr'
and 'trseg' grade components and renaming the grade component options). See
<http://www.mercurylang.org/list-archives/reviews/2014-September/017297.html>

When this came up recently Micheal (despite how he raised the issue) made a
good suggestion that we should document not just what the grade components
mean (which mostly exists in the user guide) but which combinations make
sense / are supported / are well tested.

There are have been various changes over the past year or so that
have improved this stituation:

* the user facing documentation for the experimental grades has been
hidden -- the publicly document grade components should be the ones
that are well tested.

* the compiler and mgnuc script now carry out checks of grade component
compatibility, for example:

$ mmc --grade hlc.gc.stseg foo

will result in:

mercury_compile: stack segments are incompatible with the high-level C backend

(Hmmm, the word "error" really ought to appear in that message ...)

* the usage message and user's guide were updated to say which
combinations are allowed (where that wasn't already stated).

Post by Paul Bone
This was one of the reasons I setup the wiki. There were a bunch of reasons
but this was the one that actually got me to do something. It has has very
few contributions but zero spam, so I think that there's still potential
there. Anyway, my point is that some of the "too many grades" solution is
orthogonal to making a release.

The main thing that remains to be done is to add new grade component
options. The reason we want to do that is so that we can deprecate the
existing options. We want to do that so we can change, for example,
'--debug' to means "choose the best available installed debug grade" intead of
"add the debug grade component to the current grade".

Julien.

[Mark Brown]

Post by Julien Fischer

Post by Paul Bone

Post by Julien Fischer
There was a list of things we wanted to do to address the "too many grades"
problem. Some of these have been done (e.g. hiding user facing documentation
for experimental grade components), others haven't yet (e.g. merging the 'tr'
and 'trseg' grade components and renaming the grade component options).
See
<http://www.mercurylang.org/list-archives/reviews/2014-September/017297.html>

When this came up recently Micheal (despite how he raised the issue) made a
good suggestion that we should document not just what the grade components
mean (which mostly exists in the user guide) but which combinations make
sense / are supported / are well tested.

This doesn't address the key problem, which is that users need to know
anything at all about what the grade components mean when they are
installing Mercury. We know the common use cases for installation;
most users should only have to pick which one they want. E.g.:

- user who wants to compile some code they have found
- developer who wants profilers, debugger, etc
- developer who knows exactly the set of grades they want
- sysad who wants everything because they don't want to be bothered by
follow-up requests
- user who wants to bootstrap from ROTD

The configure script ought to be able to decide the set of grades
based on this choice, perhaps with some additional tweaks available.

Post by Julien Fischer
The main thing that remains to be done is to add new grade component
options. The reason we want to do that is so that we can deprecate the
existing options. We want to do that so we can change, for example,
'--debug' to means "choose the best available installed debug grade" intead of
"add the debug grade component to the current grade".

Exactly. With options like these, and a configure script as above,
most users won't need to learn the details of grade components.

I think the deprecation of the existing options should be announced
right away - i.e., tell users the options will be changing meaning
soon.

Cheers,
Mark.

[Julien Fischer]

Hi Mark,

Post by Mark Brown

Post by Paul Bone

Post by Julien Fischer
There was a list of things we wanted to do to address the "too many grades"
problem. Some of these have been done (e.g. hiding user facing documentation
for experimental grade components), others haven't yet (e.g. merging the 'tr'
and 'trseg' grade components and renaming the grade component options).
See
<http://www.mercurylang.org/list-archives/reviews/2014-September/017297.html>

When this came up recently Micheal (despite how he raised the issue) made a
good suggestion that we should document not just what the grade components
mean (which mostly exists in the user guide) but which combinations make
sense / are supported / are well tested.

This doesn't address the key problem, which is that users need to know
anything at all about what the grade components mean when they are
installing Mercury. We know the common use cases for installation;
- user who wants to compile some code they have found

That rather depends on what code they have "found" ;-)

Even in the absence of things like debugging and profiling that may be
quite few grades.

Post by Mark Brown
- developer who wants profilers, debugger, etc
- developer who knows exactly the set of grades they want
- sysad who wants everything because they don't want to be bothered by
follow-up requests
- user who wants to bootstrap from ROTD
The configure script ought to be able to decide the set of grades
based on this choice, perhaps with some additional tweaks available.

We could add a new configure option to select a (mostly) pre-determined
set of grades, for example:

--with-installation-profile = {default,debug,all,minimal}

where the profile names correspond to the the first, second, fourth
and fifth cases above. (The above names are for illustration only,
I realise they're not great.)

The second point is handled by the existing '--enable-libgrades' option.

To an extent this just shifts the problem around, "what grades do I
install" becomes "what profile to I select"?

Julien.

[Zoltan Somogyi]

Post by Julien Fischer
We could add a new configure option to select a (mostly) pre-determined
--with-installation-profile = {default,debug,all,minimal}
where the profile names correspond to the the first, second, fourth
and fifth cases above. (The above names are for illustration only,
I realise they're not great.)
The second point is handled by the existing '--enable-libgrades' option.
To an extent this just shifts the problem around, "what grades do I
install" becomes "what profile to I select"?

But the second problem will almost certainly be a much easier problem
for most people to solve, so I think the above is a very good idea.

A couple of notes on it. First: as you say a "(mostly) predetermined set":
the results of autoconfiguration will affect whether we include e.g.
none.gc.debug.stseg or asm_fast.gc.debug.stseg as the debugging grade.
Second: when we print the set of libgrades at the end of the output of configure,
and this list was NOT controlled by --enable-libgrades, we should include
a message about --with-installation-profile and values it can be given.

Zoltan.

[Julien Fischer]

Post by Zoltan Somogyi

Post by Julien Fischer
We could add a new configure option to select a (mostly) pre-determined
--with-installation-profile = {default,debug,all,minimal}
where the profile names correspond to the the first, second, fourth
and fifth cases above. (The above names are for illustration only,
I realise they're not great.)
The second point is handled by the existing '--enable-libgrades' option.
To an extent this just shifts the problem around, "what grades do I
install" becomes "what profile to I select"?

But the second problem will almost certainly be a much easier problem
for most people to solve, so I think the above is a very good idea.

When that's done it's probably also worth undocumenting (or possibly
just deleting) many of the existing --{enable,disable}-*-grades
configure options.

I'll post a proposed set of installation profiles later.

Post by Zoltan Somogyi
the results of autoconfiguration will affect whether we include e.g.
none.gc.debug.stseg or asm_fast.gc.debug.stseg as the debugging grade.

Yes.

Post by Zoltan Somogyi
Second: when we print the set of libgrades at the end of the output of configure,
and this list was NOT controlled by --enable-libgrades, we should include
a message about --with-installation-profile and values it can be given.

Ok.

Julien.

[Mark Brown]

Post by Julien Fischer
Hi Mark,

Post by Mark Brown
- user who wants to compile some code they have found

That rather depends on what code they have "found" ;-)
Even in the absence of things like debugging and profiling that may be
quite few grades.

I should say code that works in any grade. So for this case I'm saying
only one grade needs to be installed, and it doesn't matter which.

Post by Julien Fischer
To an extent this just shifts the problem around, "what grades do I
install" becomes "what profile to I select"?

Yes, that's the point - it's a question that users can answer by
themselves without having to look up any jargon. ;-)

Cheers,
Mark.

[Zoltan Somogyi]

Post by Julien Fischer
There was a list of things we wanted to do to address the "too many grades"
problem. Some of these have been done (e.g. hiding user facing documentation
for experimental grade components), others haven't yet (e.g. merging the 'tr'
and 'trseg' grade components and renaming the grade component options). See
<http://www.mercurylang.org/list-archives/reviews/2014-September/017297.html>

Actually, I was thinking about grade options earlier today, prompted by the same
concern. I think I will have a proposal worked out in a day or two.

Zoltan.

[Zoltan Somogyi]

Post by Zoltan Somogyi
Actually, I was thinking about grade options earlier today, prompted by the same
concern. I think I will have a proposal worked out in a day or two.

My proposal is attached. At the moment, it is a description
of an approach, with some details missing, because it makes sense
to work out those details only if there is agreement on the basic
approach. I intend this proposal to start a discussion on its approach.

If someone wants to turn the proposal into a wiki, go ahead.

Zoltan.

[Paul Bone]

Post by Zoltan Somogyi

Post by Zoltan Somogyi
Actually, I was thinking about grade options earlier today, prompted by the same
concern. I think I will have a proposal worked out in a day or two.

My proposal is attached. At the moment, it is a description
of an approach, with some details missing, because it makes sense
to work out those details only if there is agreement on the basic
approach. I intend this proposal to start a discussion on its approach.
If someone wants to turn the proposal into a wiki, go ahead.

Thank you. I like the general idea here and believe that it will be worth
while. It represents a good compromise between the current power we have in
choosing so many build options, and making things simpler. I also think
that, apart from some double maintenance and a related problem of "what
grades do I install?" it is the simplest (least effort to implement)
proposal that could be made given our current implementation.

For things like this, which are mostly of interest to developers, the
discussions repository is a better place for it than the wiki. How it works
from the user's perspective is probably a good submission for the wiki.

Post by Zoltan Somogyi
The idea is to let a user specify via an option that he/she would like
to choose a specific value for a specific solver variable. The system would
1 take the set of such choices expressed on the command line
2 if those choices are inconsistent, then
3 report an error for each source of inconsistency
4 else
6 compute the set of values of this solver variable that are
7 consistent with the values of the already-bound variables
8 if this set contains just one value
9 bind the variable to this value
10 else
11 bind the variable to the value in the set that
the user is most likely to implicitly want
(in the table below, the default is the most preferred value)

It looks like lines 8-9 are intended to implement a propagation step. In
that case I would prefer to use a while loop (line 5), and propagate to
_any_ variable with a singleton set of "consistent values".

For example:

take the set of such choices expressed on the command line
if those choices are inconsistent, then
report an error for each source of inconsistency
else
5 while there are unbound variables
for each so-far unbound variable (any order):
compute the set of values of this solver variable that are
consistent with the values of the already-bound variables
if this set contains just one value
bind the variable to this value
break out of both loops (back to line 5)
choose a variable (using "chosen order" as in Zoltan's line 5
bind the variable to the value in the set that
the user is most likely to implicitly want
(in the table below, the default is the most preferred value)

This ensures that if the user specifies TAR=java, then CL=high will always
be set (via propagation) before a choice is made on a variable such as REG.
(REG=n will be propagated soon after.)

Post by Zoltan Somogyi
--------------------------------------------------------
NOTE: This includes only the user-visible solver variables. There will be
some non-user-visible ones as well, such as high tag vs low tag bits
vs no tag bits, and --num-tag-bits.
XXXs means something is missing, and will need to be added.
??? marks a decision that I (zs) think should be reviewed.
CL.hlc: high level code
CL.llc: low level code (requires TAR.c)
DL.hld: high level data (requires CL.hlc)
DL.lld: low level data
(default is DL.lld)
TAR.c: C
TAR.cs: C# (requires CL.hlc)
TAR.j: Java (requires CL.hlc)
TAR.e: Erlang (requires CL.hlc)
(default is TAR.c)

I think that "j" is the name of a language. We may want to avoid confusion
and call this "java".

Post by Zoltan Somogyi
NEST.n: no gcc nested functions
NEST.y: gcc nested functions (requires CL.hlc and TAR.c)
(default is NEST.n)
REG.n: no gcc global registers
REG.y: gcc global registers (requires CL.llc and TAR.c)
(default is REG.y if supported on platform)
GOTO.n: no gcc nonlocal gotos
GOTO.y: gcc nonlocal gotos (requires CL.llc and TAR.c)
(default is GOTO.y if supported on platform)
ASM.n: no gcc asm labels
ASM.y: gcc asm labels (requires CL.b and GOTO.y)
(default is ASM.y if supported on platform)

In your "chosen order" we may have to be careful with these variables, or
even choose the order dynamically or add backtracking.

If a user has not specified anything and a their platform supports REG but
not GOTO, and REG=y is selected first, then that would propagate CL=llc.
Unfortunately they'll end up with a grade like reg.gc when they could have
had a faster grade like hlc.gc. This can probably be fixed by always
choosing CL before REG, GOTO or ASM.

Post by Zoltan Somogyi
SS.n: no stack segments
SS.y: stack segments (requires CL.llc)
(default is SS.y ???)
XXX we should make MR_EXTEND_STACKS_WHEN_NEEDED undocumented
TSAFE.n: no thread safe
TSAFE.y: thread safe
(default is TSAFE.n)
GC.no: no gc
GC.bdw: boehm gc
GC.bdwd: boehm gc debug
GC.nat: native gc (requires CL.hlc and TAR.{java,erlang,XXX}
GC.acc: accurate gc (requires CL.hlc and TAR.c)
GC.rafe: becket history gc (requires CL.hlc and XXX)
(default is GC.bdw)

While GC=nat requires TAR=java, erlang or C#. I think it's more natural to
make this propagate the other way. So TAR=java implies GC=nat. (Both are
logically true, just from a user's perspective.)

Post by Zoltan Somogyi
DEEP.n: no deep profiling
DEEP.y: deep profiling (requires CL.llc and TIME.n and CALLS.n and MEM.n)
(default is DEEP.n)
TIME.n: no gprof time profiling
TIME.y: gprof time profiling (requires CL.llc and DEEP.n)
(default is TIME.n)
CALL.n: no gprof call profiling
CALL.y: gprof call profiling (requires CL.llc and DEEP.n and XXX)
(default is CALL.n)
MEM.n: no gprof memory profiling
MEM.y: gprof memory profiling (requires CL.llc and DEEP.n and XXX)
(default is MEM.n)
SCOPE.n: no threadscope profiling
SCOPE.c: threadscope profiling (requires XXX)
(default is SCOPE.n)

Requires TAR=c, CL=llc and TSAFE=y. It should probably be relaxed to also
work with high level C.

It is very much unsupported at the moment. The graphical threadscope tool
no-longer recognizes our profiles. so supporting it without writing our own
viewer is useless.

Post by Zoltan Somogyi
SIZE.n: no term size profiling
SIZE.c: term size profiling cells (requires CL.llc)
SIZE.w: term size profiling words (requires CL.llc)
(default is SIZE.n)
TR.n: no trailing
TR.y: trailing (requires MM.n)
(default is TR.n)
MM.n: no minimal model tabling
MM.c: stack copy mm, no debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n)
MM.cd: stack copy mm, debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n) (not yet documented)
MM.o: own stack mm, no debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n) (not yet documented)
MM.od: own stack mm, debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n) (not yet documented)
(default is MM.n)
SPF.n: no single prec float
SPF.y: single prec float
(default is SPF.n)

Requires TAR=c?

Post by Zoltan Somogyi
DBG.n: no debugging
DBG.d: debugging (requires CL.llc ???)
DBG.dd: declarative debugging (requires CL.llc)
DBG.s: ss debugging (requires CL.hlc ???)
TSCOPE.n: no threadscope
TSCOPE.y: threadscope (requires XXX)

You have this as SCOPE above. TSCOPE is probably a better name.

I started work to rename this completely, after realising that support for
Mercury in the threadscope viewer would frequently be broken.

[Zoltan Somogyi]

Post by Paul Bone
It looks like lines 8-9 are intended to implement a propagation step. In
that case I would prefer to use a while loop (line 5), and propagate to
_any_ variable with a singleton set of "consistent values".
take the set of such choices expressed on the command line
if those choices are inconsistent, then
report an error for each source of inconsistency
else
5 while there are unbound variables
compute the set of values of this solver variable that are
consistent with the values of the already-bound variables
if this set contains just one value
bind the variable to this value
break out of both loops (back to line 5)
choose a variable (using "chosen order" as in Zoltan's line 5
bind the variable to the value in the set that
the user is most likely to implicitly want
(in the table below, the default is the most preferred value)
This ensures that if the user specifies TAR=java, then CL=high will always
be set (via propagation) before a choice is made on a variable such as REG.
(REG=n will be propagated soon after.)

The two algorithms are equivalent. If a solver variable is set by the user
(e.g. TAR=java), then no assignment can be consistent unless it includes
all the settings implied by that (in this case, CL=high). Whether you record
CL=high immediately or when the loop gets to looking at the possible values
of CL does not matter.

Post by Paul Bone

Post by Zoltan Somogyi
TAR.c: C
TAR.cs: C# (requires CL.hlc)
TAR.j: Java (requires CL.hlc)
TAR.e: Erlang (requires CL.hlc)
(default is TAR.c)

I think that "j" is the name of a language. We may want to avoid confusion
and call this "java".

You are taking this too literally. This is a design description, not code.
The reason why I kept the names short was to fit the constraints involving
them into 80 columns. The actual names of the mutables will be more verbose,
and in any case, no user will ever see them. They will see only the names
of the options, and I was explicit about the fact that I am not yet proposing
any names for THEM.

Post by Paul Bone
even choose the order dynamically or add backtracking.

I don't think that will be needed. I certainly hope it won't, because
implementing it in sh would be a bitch :-(

Post by Paul Bone
If a user has not specified anything and a their platform supports REG but
not GOTO, and REG=y is selected first, then that would propagate CL=llc.
Unfortunately they'll end up with a grade like reg.gc when they could have
had a faster grade like hlc.gc. This can probably be fixed by always
choosing CL before REG, GOTO or ASM.

Or by having the choice of CL (in the rare case when it isn't implied by other
options) be influenced by the availability of registers, labels etc on the platform.

Post by Paul Bone

Post by Zoltan Somogyi
GC.no: no gc
GC.bdw: boehm gc
GC.bdwd: boehm gc debug
GC.nat: native gc (requires CL.hlc and TAR.{java,erlang,XXX}
GC.acc: accurate gc (requires CL.hlc and TAR.c)
GC.rafe: becket history gc (requires CL.hlc and XXX)
(default is GC.bdw)

While GC=nat requires TAR=java, erlang or C#. I think it's more natural to
make this propagate the other way. So TAR=java implies GC=nat. (Both are
logically true, just from a user's perspective.)

Again, you are reading this proposal as being more concrete than it was
intended to be. I am only describing constraints; I am NOT proposing
ANYTHING about how they should be solved. The XXXs and ???s show
I am not even sure about the exact set of constraints we need.

Post by Paul Bone

Post by Zoltan Somogyi
TSCOPE.n: no threadscope
TSCOPE.y: threadscope (requires XXX)

You have this as SCOPE above. TSCOPE is probably a better name.

Again, it this point in time it does not matter.

Post by Paul Bone
I started work to rename this completely, after realising that support for
Mercury in the threadscope viewer would frequently be broken.

Rename what in what?

Zoltan.

[Paul Bone]

Post by Zoltan Somogyi

Post by Paul Bone
It looks like lines 8-9 are intended to implement a propagation step. In
that case I would prefer to use a while loop (line 5), and propagate to
_any_ variable with a singleton set of "consistent values".
take the set of such choices expressed on the command line
if those choices are inconsistent, then
report an error for each source of inconsistency
else
5 while there are unbound variables
compute the set of values of this solver variable that are
consistent with the values of the already-bound variables
if this set contains just one value
bind the variable to this value
break out of both loops (back to line 5)
choose a variable (using "chosen order" as in Zoltan's line 5
bind the variable to the value in the set that
the user is most likely to implicitly want
(in the table below, the default is the most preferred value)
This ensures that if the user specifies TAR=java, then CL=high will always
be set (via propagation) before a choice is made on a variable such as REG.
(REG=n will be propagated soon after.)

The two algorithms are equivalent. If a solver variable is set by the user
(e.g. TAR=java), then no assignment can be consistent unless it includes
all the settings implied by that (in this case, CL=high). Whether you record
CL=high immediately or when the loop gets to looking at the possible values
of CL does not matter.

What if the loop records REG=y before it gets to CL? Or will the "choosen
order" always handle those situations?

Post by Zoltan Somogyi

Post by Paul Bone

Post by Zoltan Somogyi
TAR.c: C
TAR.cs: C# (requires CL.hlc)
TAR.j: Java (requires CL.hlc)
TAR.e: Erlang (requires CL.hlc)
(default is TAR.c)

I think that "j" is the name of a language. We may want to avoid confusion
and call this "java".

You are taking this too literally. This is a design description, not code.
The reason why I kept the names short was to fit the constraints involving
them into 80 columns. The actual names of the mutables will be more verbose,
and in any case, no user will ever see them. They will see only the names
of the options, and I was explicit about the fact that I am not yet proposing
any names for THEM.

sorry.

Post by Zoltan Somogyi

Post by Paul Bone
even choose the order dynamically or add backtracking.

I don't think that will be needed. I certainly hope it won't, because
implementing it in sh would be a bitch :-(

Yeah, I was worried the first time I read "solver variable".

Post by Zoltan Somogyi

Post by Paul Bone
If a user has not specified anything and a their platform supports REG but
not GOTO, and REG=y is selected first, then that would propagate CL=llc.
Unfortunately they'll end up with a grade like reg.gc when they could have
had a faster grade like hlc.gc. This can probably be fixed by always
choosing CL before REG, GOTO or ASM.

Or by having the choice of CL (in the rare case when it isn't implied by other
options) be influenced by the availability of registers, labels etc on the platform.

Yep.

Post by Zoltan Somogyi

Post by Paul Bone
I started work to rename this completely, after realising that support for
Mercury in the threadscope viewer would frequently be broken.

Rename what in what?

Rename --threadscope and the threadscope grade component to
--parallel-profiling. Since it wouldn't be compatible with ThreadScope and
this name is more descriptive.

[Zoltan Somogyi]

Post by Paul Bone

Post by Zoltan Somogyi

Post by Zoltan Somogyi
take the set of such choices expressed on the command line
if those choices are inconsistent, then
report an error for each source of inconsistency
else
5 while there are unbound variables
compute the set of values of this solver variable that are
consistent with the values of the already-bound variables
if this set contains just one value
bind the variable to this value
break out of both loops (back to line 5)
choose a variable (using "chosen order" as in Zoltan's line 5
bind the variable to the value in the set that
the user is most likely to implicitly want
(in the table below, the default is the most preferred value)
This ensures that if the user specifies TAR=java, then CL=high will always
be set (via propagation) before a choice is made on a variable such as REG.
(REG=n will be propagated soon after.)

The two algorithms are equivalent. If a solver variable is set by the user
(e.g. TAR=java), then no assignment can be consistent unless it includes
all the settings implied by that (in this case, CL=high). Whether you record
CL=high immediately or when the loop gets to looking at the possible values
of CL does not matter.

What if the loop records REG=y before it gets to CL? Or will the "choosen
order" always handle those situations?

In this respect, the order does not matter. What matters is this part
of my original algorithm:

compute the set of values of this solver variable that are
consistent with the values of the already-bound variables

The consistency check here does the job of the propagation steps you want,
and may (or may not) be implemented via propagation.

If some option set by the user requires CL=high, then y is not a possible
value for REG (since REG=y requires CL=low), even if you consider REG before CL.

Post by Paul Bone

Post by Zoltan Somogyi
You are taking this too literally. This is a design description, not code.
The reason why I kept the names short was to fit the constraints involving
them into 80 columns. The actual names of the mutables will be more verbose,
and in any case, no user will ever see them. They will see only the names
of the options, and I was explicit about the fact that I am not yet proposing
any names for THEM.

sorry.

No harm done.

Post by Paul Bone

Post by Zoltan Somogyi

Post by Zoltan Somogyi
even choose the order dynamically or add backtracking.

I don't think that will be needed. I certainly hope it won't, because
implementing it in sh would be a bitch :-(

Yeah, I was worried the first time I read "solver variable".

That is one reason why this hasn't been done before.

The fact that Mercury precedes HAL is another :-)

Zoltan.

[Paul Bone]

Post by Zoltan Somogyi

Post by Zoltan Somogyi
Actually, I was thinking about grade options earlier today, prompted by the same
concern. I think I will have a proposal worked out in a day or two.

My proposal is attached. At the moment, it is a description
of an approach, with some details missing, because it makes sense
to work out those details only if there is agreement on the basic
approach. I intend this proposal to start a discussion on its approach.
If someone wants to turn the proposal into a wiki, go ahead.
Zoltan.

A related point of confusion are features supported by different grades.

The grades proposal might want to consider information from feature sets
found in a project's source code:

http://www.mercurylang.org/information/doc-latest/mercury_ref/Feature-sets.html#Feature-sets

Trailing is a nice simple one. It corresponds directly to the trailing
variable in Zoltan's proposal.

A tricker one is parallel conjunction, it does not correspond directly to
any option. It could be viewed as a separate variable that only serves to
constrain other variables. It implies thread safe and low-level C.

A very tricky option is exactly how concurrency works. This is something
I've been asked before and badly need to document. (This is a perfect item
for the wiki incidentally).

Low level C:
Thread safe:
Some concurrency, but if you do IO you can block a whole Mercury
engine, which might be your only engine.
Non thread safe:
I think concurrency is currently broken. I don't think anyone has
tested or used it. I believe that it was intended to work.
High level C:
Thread safe:
It works, but you get one pthread per Mercury thread, which might be
what you want, or it might not.
Non-thread safe:
Doesn't work.
Java & C#
Same as high level C, thread safe.

If you want parallel conjunctions and concurrency that behaves properly with
IO then your only option is to use low level C with spawn_native. But
that's not ideal.

The other solution is to implement proper behaviour with IO in low-level
grades, and parallel conjunctions in high level grades :-P

Anyway, we could either delay this until after Zoltan's idea or something
else is accepted and implemented. Or create options such as --parallel-conj
that add sufficient constraints to Zoltan's proposal to choose a grade for
the user in those cases.

[Julien Fischer]

Hi Zoltan,

Post by Zoltan Somogyi

Post by Zoltan Somogyi
Actually, I was thinking about grade options earlier today, prompted by the same
concern. I think I will have a proposal worked out in a day or two.

My proposal is attached. At the moment, it is a description
of an approach, with some details missing, because it makes sense
to work out those details only if there is agreement on the basic
approach. I intend this proposal to start a discussion on its approach.
If someone wants to turn the proposal into a wiki, go ahead.

Some comments:

Incidentally, I suggest you add the revised document to compiler/notes.

...

Post by Zoltan Somogyi
At the moment, we have three pieces of code for computing the grade from
options. The Mercury version of this code is used by the compiler, and is
now in compiler/compute_grade.m. The C version, used by e.g. mgnuc and ml,
is in scripts/*_grade.sh-subr. The algorithm above is too complex to implement
in sh, so I propose that it be put into a small library containing nothing
but the code that takes a list of grade-related options, and returns
the selected solution of the constraint problem they represent. The code
of compute_grade.m would invoke this small library, and so would a new
standalone Mercury program, named e.g. compute_grade_for_sh.m, that
would be invoked from a new sh script, named e.g. compute_grade.sh-subr.

Won't this be a problem when building from the source distribution since the
small Mercury program would need to be compiled from the pre-generated .c files
and doing so requires both the mgnuc and ml scripts? Also, in the source
distribution we ideally want the canonical_grade script to be available so that
if the user supplies a list of grades to configure (e.g. via
--enable-libgrades), we can sanity check them when configure is run.

Post by Zoltan Somogyi
The latter would replace the existing script/*grade.sh-subr, and probably
also scripts/canonical grade. The third copy is in runtime/mercury_grade.h,
and it will continue to need double maintenance with the Mercury code.
If the use of a Mercury by e.g. canonical_grade would be a problem,
we could try continuing to use sh code to compute the grade. The algorithm
above should *just about* be doable in sh, even though it would not be
pretty sight.

mmc (when invoked by itself) and mmc --make require will the new grade related
machinery, since they are what we are intend end users of the system to use.
I don't think we need to do anything with mgnuc / ml (mmake), since at this
point they are primarily tools for building the Mercury system itself.

Post by Zoltan Somogyi
If there is broad agreement that something like this is the way to go,
then it will make sense to make concrete proposals for the concrete names
of the options that give specific values to specific solver variables.

The proposal is fine by me, with the reservation regarding mgnuc / ml etc.

Post by Zoltan Somogyi
--------------------------------------------------------
NOTE: This includes only the user-visible solver variables. There will be
some non-user-visible ones as well, such as high tag vs low tag bits
vs no tag bits, and --num-tag-bits.
XXXs means something is missing, and will need to be added.
??? marks a decision that I (zs) think should be reviewed.
CL.hlc: high level code
CL.llc: low level code (requires TAR.c)

Strictly speaking, the Erlang backend is neither of those.

Post by Zoltan Somogyi
DL.hld: high level data (requires CL.hlc)
DL.lld: low level data
(default is DL.lld)
TAR.c: C
TAR.cs: C# (requires CL.hlc)
TAR.j: Java (requires CL.hlc)
TAR.e: Erlang (requires CL.hlc)
(default is TAR.c)
NEST.n: no gcc nested functions
NEST.y: gcc nested functions (requires CL.hlc and TAR.c)
(default is NEST.n)
REG.n: no gcc global registers
REG.y: gcc global registers (requires CL.llc and TAR.c)
(default is REG.y if supported on platform)
GOTO.n: no gcc nonlocal gotos
GOTO.y: gcc nonlocal gotos (requires CL.llc and TAR.c)
(default is GOTO.y if supported on platform)
ASM.n: no gcc asm labels
ASM.y: gcc asm labels (requires CL.b and GOTO.y)
(default is ASM.y if supported on platform)
SS.n: no stack segments
SS.y: stack segments (requires CL.llc)
(default is SS.y ???)
XXX we should make MR_EXTEND_STACKS_WHEN_NEEDED undocumented
TSAFE.n: no thread safe
TSAFE.y: thread safe
(default is TSAFE.n)
GC.no: no gc
GC.bdw: boehm gc
GC.bdwd: boehm gc debug
GC.nat: native gc (requires CL.hlc and TAR.{java,erlang,XXX}

and TAR.cs.

Post by Zoltan Somogyi
GC.acc: accurate gc (requires CL.hlc and TAR.c)
GC.rafe: becket history gc (requires CL.hlc and XXX)

Ralph's GC is TAR.c.

Post by Zoltan Somogyi
(default is GC.bdw)
DEEP.n: no deep profiling
DEEP.y: deep profiling (requires CL.llc and TIME.n and CALLS.n and MEM.n)
(default is DEEP.n)
TIME.n: no gprof time profiling
TIME.y: gprof time profiling (requires CL.llc and DEEP.n)
(default is TIME.n)
CALL.n: no gprof call profiling
CALL.y: gprof call profiling (requires CL.llc and DEEP.n and XXX)

Requires DEEP.n and TAR.c (not CL.llc). Call profiling also works with the
high-level C backend.

Post by Zoltan Somogyi
(default is CALL.n)
MEM.n: no gprof memory profiling
MEM.y: gprof memory profiling (requires CL.llc and DEEP.n and XXX)
(default is MEM.n)

Ditto regarding the high-level C backend.

Post by Zoltan Somogyi
SCOPE.n: no threadscope profiling
SCOPE.c: threadscope profiling (requires XXX)
(default is SCOPE.n)
SIZE.n: no term size profiling
SIZE.c: term size profiling cells (requires CL.llc)
SIZE.w: term size profiling words (requires CL.llc)
(default is SIZE.n)
TR.n: no trailing
TR.y: trailing (requires MM.n)
(default is TR.n)
MM.n: no minimal model tabling
MM.c: stack copy mm, no debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n)
MM.cd: stack copy mm, debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n) (not yet documented)
MM.o: own stack mm, no debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n) (not yet documented)
MM.od: own stack mm, debug (requires CL.llc and (GC.bdw or GC.bdwd) and TR.n) (not yet documented)
(default is MM.n)
SPF.n: no single prec float
SPF.y: single prec float
(default is SPF.n)

SPF.y requires TAR.c.

Post by Zoltan Somogyi
DBG.n: no debugging
DBG.d: debugging (requires CL.llc ???)

I'm not sure why you have ??? there.

Post by Zoltan Somogyi
DBG.dd: declarative debugging (requires CL.llc)
DBG.s: ss debugging (requires CL.hlc ???)

ssdebug should work with any of the target languages, so the only requirement
is CL.hlc.

Julien.

[Mark Brown]

Hi Julien,

Post by Julien Fischer
Hi Zoltan,

Post by Zoltan Somogyi
At the moment, we have three pieces of code for computing the grade from
options. The Mercury version of this code is used by the compiler, and is
now in compiler/compute_grade.m. The C version, used by e.g. mgnuc and ml,
is in scripts/*_grade.sh-subr. The algorithm above is too complex to implement
in sh, so I propose that it be put into a small library containing nothing
but the code that takes a list of grade-related options, and returns
the selected solution of the constraint problem they represent. The code
of compute_grade.m would invoke this small library, and so would a new
standalone Mercury program, named e.g. compute_grade_for_sh.m, that
would be invoked from a new sh script, named e.g. compute_grade.sh-subr.

Won't this be a problem when building from the source distribution since the
small Mercury program would need to be compiled from the pre-generated .c files
and doing so requires both the mgnuc and ml scripts?

The scripts will only really need to call this program if high-level
options for selecting the grade are actually used. If the grade is
given explicitly and in canonical form then that won't be needed, so
in this case the scripts can be run before the program is built.

So configure would have two jobs:
- Without using the compute_grade library, figure out a single grade
that will definitely work and can be used to build the compute_grade
library and the standalone programs. This would surely be the same
grade as will be used to build the compiler.
- Determine the platform dependent constraints that the compute_grade
library would use.

Post by Julien Fischer
Also, in the source
distribution we ideally want the canonical_grade script to be available so that
if the user supplies a list of grades to configure (e.g. via
--enable-libgrades), we can sanity check them when configure is run.

The sanity checks could be done at the start of the make step, which
is not that much later than currently.

Post by Julien Fischer

Post by Zoltan Somogyi
The latter would replace the existing script/*grade.sh-subr, and probably
also scripts/canonical grade. The third copy is in
runtime/mercury_grade.h,
and it will continue to need double maintenance with the Mercury code.
If the use of a Mercury by e.g. canonical_grade would be a problem,
we could try continuing to use sh code to compute the grade. The algorithm
above should *just about* be doable in sh, even though it would not be
pretty sight.

mmc (when invoked by itself) and mmc --make require will the new grade related
machinery, since they are what we are intend end users of the system to use.
I don't think we need to do anything with mgnuc / ml (mmake), since at this
point they are primarily tools for building the Mercury system itself.

They will need to be able to compile in the grade determined by
configure for compiling the compiler, compute_grade library, and the
standalone programs. The "double maintenance" issue is that this
grade, on all platforms, has to be the same one that is selected via
an unconstrained search using the new method. Everything else will go
via the compute_grade library.

Does that address your reservations?

Cheers,
Mark.

[Julien Fischer]

Hi Mark,

Post by Mark Brown

Post by Julien Fischer
Hi Zoltan,

Post by Zoltan Somogyi
At the moment, we have three pieces of code for computing the grade from
options. The Mercury version of this code is used by the compiler, and is
now in compiler/compute_grade.m. The C version, used by e.g. mgnuc and ml,
is in scripts/*_grade.sh-subr. The algorithm above is too complex to implement
in sh, so I propose that it be put into a small library containing nothing
but the code that takes a list of grade-related options, and returns
the selected solution of the constraint problem they represent. The code
of compute_grade.m would invoke this small library, and so would a new
standalone Mercury program, named e.g. compute_grade_for_sh.m, that
would be invoked from a new sh script, named e.g. compute_grade.sh-subr.

Won't this be a problem when building from the source distribution since the
small Mercury program would need to be compiled from the pre-generated .c files
and doing so requires both the mgnuc and ml scripts?

The scripts will only really need to call this program if high-level
options for selecting the grade are actually used. If the grade is
given explicitly and in canonical form then that won't be needed, so
in this case the scripts can be run before the program is built.
- Without using the compute_grade library, figure out a single grade
that will definitely work and can be used to build the compute_grade
library and the standalone programs. This would surely be the same
grade as will be used to build the compiler.

It will necessarily have to be that grade ;-) You won't have a runtime
or standard library to link it against otherwise.

Post by Mark Brown
- Determine the platform dependent constraints that the compute_grade
library would use.

Post by Julien Fischer
Also, in the source
distribution we ideally want the canonical_grade script to be available so that
if the user supplies a list of grades to configure (e.g. via
--enable-libgrades), we can sanity check them when configure is run.

The sanity checks could be done at the start of the make step, which
is not that much later than currently.

Ideally not, since that introduces a disconnect between the action of
running ./configure and the point at which you get the error message.

Post by Mark Brown

Post by Julien Fischer

Post by Zoltan Somogyi
The latter would replace the existing script/*grade.sh-subr, and probably
also scripts/canonical grade. The third copy is in
runtime/mercury_grade.h,
and it will continue to need double maintenance with the Mercury code.
If the use of a Mercury by e.g. canonical_grade would be a problem,
we could try continuing to use sh code to compute the grade. The algorithm
above should *just about* be doable in sh, even though it would not be
pretty sight.

mmc (when invoked by itself) and mmc --make require will the new grade related
machinery, since they are what we are intend end users of the system to use.
I don't think we need to do anything with mgnuc / ml (mmake), since at this
point they are primarily tools for building the Mercury system itself.

They will need to be able to compile in the grade determined by
configure for compiling the compiler, compute_grade library, and the
standalone programs. The "double maintenance" issue is that this
grade, on all platforms, has to be the same one that is selected via
an unconstrained search using the new method. Everything else will go
via the compute_grade library.
Does that address your reservations?

I have two issues here:

1. Should you be able to use the new method of specifying a grade when
building the Mercury system itself? I'm saying no: for the source
distribution you don't have a choice of grade**, non-developers are better
off going with what configure decides and developers ought to know how
to specify a grade directly.

(** these days it must be hlc.gc.pregen)

2. Do we continue to support mmake (and its attendent shell scripts) as
a user level feature (as opposed to just something that builds the
Mercury system)? If not, then we don't need to support the new method
with them. (Note that mmake already lacks many features that mmc --make
has, in addition to its documented problems with sub-modules etc.)

Julien.

[Mark Brown]

Hi Julien,

Post by Julien Fischer
Hi Mark,

[Scripts] will need to be able to compile in the grade determined by
configure for compiling the compiler, compute_grade library, and the
standalone programs. The "double maintenance" issue is that this
grade, on all platforms, has to be the same one that is selected via
an unconstrained search using the new method.

Actually, that's not true; I was confusing the grade for compiling the
compiler with the default grade for compiling applications. The
unconstrained search should select the latter, if given.

So would the effect of configuring a default grade for applications
would be to set the most preferred values used in step 11 of the
search algorithm?

Post by Julien Fischer
1. Should you be able to use the new method of specifying a grade when
building the Mercury system itself? I'm saying no: for the source
distribution you don't have a choice of grade**, non-developers are better
off going with what configure decides and developers ought to know how
to specify a grade directly.

I'd like to use the new system when a bootstrapping compiler is being
used. I know how to select the grade I want from a list, but I can
never remember the correct spelling or canonical order. In practice I
use --output-libgrades a lot.

See next answer, though.

Post by Julien Fischer
2. Do we continue to support mmake (and its attendent shell scripts) as
a user level feature (as opposed to just something that builds the
Mercury system)? If not, then we don't need to support the new method
with them. (Note that mmake already lacks many features that mmc --make
has, in addition to its documented problems with sub-modules etc.)

I wouldn't consider it compulsory. It would be nice to include a user
level standalone program that simply outputs the chosen grade given
new options, which could be used with a --grade option. E.g., mgnuc
--grade `choose_grade --feature profdeep`.

Cheers,
Mark

[Mark Brown]

Hi Zoltan,

On Thu, Oct 22, 2015 at 1:45 PM, Zoltan Somogyi

Post by Zoltan Somogyi
My proposal is attached. At the moment, it is a description
of an approach, with some details missing, because it makes sense
to work out those details only if there is agreement on the basic
approach. I intend this proposal to start a discussion on its approach.

There are several related but distinct issues that have come up in
this and the other thread. It is worth listing these issues to check
also that there is agreement on the problems at hand, and to be clear
on which of these problems each proposal addresses.

1. When compiling or linking, the user needs to choose exactly one
grade. Command line options impose constraints on this choice, in
additional to the constraints as to which grades are valid, available
on the platform, and installed. If the constraints are not
satisfiable, it is up to the compiler to explain why.

2. When installing (either a user library or Mercury itself), the user
needs to choose a set of grades. Ideally this will cover any grade
that may be chosen when compiling or linking. Julien's thread is
addressing this problem.

3. With the current installation method you don't want to install too
many grades up front because of the installation time, so this is
closely connected with 2. But incremental installation - something
that was mentioned a while ago - would break this connection,
therefore installation time is a separate problem.

4. When adding a new feature that is binary incompatible hence
requires its own grade component, the developer may end up adding a
large number of grades. They need to do so in a way that minimizes the
impact on the users who are dealing with issues 1 and 2, meaning that
existing uses of command line options don't change the selection.

Your proposal addresses problems 1 and 4. I strongly agree with the
approach for the latter, and moreover I don't see any feasible
alternative.

While I agree with the approach for problem 1, including a
comprehensive and systematic form of command-line options, I'd also
like to see something even higher-level. For example, a --feature
option that chooses features such as mdb debugging, profiling, etc.
The additional challenge for such constraints would be in providing an
explanation of what to do if they are not satisfiable.

There ought to be a correspondence between the command-line options
and the features that are referred to in pragma require_feature_set,
since the concepts being expressed have much in common. This is why I
suggested the above option. I think it is also worth considering
whether the pragma should be extended to cover your proposed form of
command-line options.

Cheers,
Mark.

[Julien Fischer]

Post by Mark Brown
On Thu, Oct 22, 2015 at 1:45 PM, Zoltan Somogyi

Post by Zoltan Somogyi
My proposal is attached. At the moment, it is a description
of an approach, with some details missing, because it makes sense
to work out those details only if there is agreement on the basic
approach. I intend this proposal to start a discussion on its approach.

...

Post by Mark Brown
3. With the current installation method you don't want to install too
many grades up front because of the installation time, so this is
closely connected with 2. But incremental installation - something
that was mentioned a while ago - would break this connection,
therefore installation time is a separate problem.

For the record, I did post a diff that allowed incremental installation
of library grades (on most platforms) here:
<http://www.mercurylang.org/list-archives/reviews/2014-September/017263.html>

...

Post by Mark Brown
While I agree with the approach for problem 1, including a
comprehensive and systematic form of command-line options, I'd also
like to see something even higher-level. For example, a --feature
option that chooses features such as mdb debugging, profiling, etc.
The additional challenge for such constraints would be in providing an
explanation of what to do if they are not satisfiable.
There ought to be a correspondence between the command-line options
and the features that are referred to in pragma require_feature_set,
since the concepts being expressed have much in common. This is why I
suggested the above option. I think it is also worth considering
whether the pragma should be extended to cover your proposed form of
command-line options.

If such a correspondence is made, then we should also support the
command line options --libgrades-{exclude,include}-feature in addition
to the existing --libgrades-{exclude,include}-component.

Julien.

[Mark Brown]

Post by Julien Fischer

Post by Mark Brown
3. With the current installation method you don't want to install too
many grades up front because of the installation time, so this is
closely connected with 2. But incremental installation - something
that was mentioned a while ago - would break this connection,
therefore installation time is a separate problem.

For the record, I did post a diff that allowed incremental installation
<http://www.mercurylang.org/list-archives/reviews/2014-September/017263.html>

That's why I mentioned it specifically as a separate problem. :-)

Can the incremental installation be made to work with the (additive)
profiles proposed in the other thread?

Cheers,
Mark.

[Julien Fischer]

Post by Mark Brown

Post by Julien Fischer

Post by Mark Brown
3. With the current installation method you don't want to install too
many grades up front because of the installation time, so this is
closely connected with 2. But incremental installation - something
that was mentioned a while ago - would break this connection,
therefore installation time is a separate problem.

For the record, I did post a diff that allowed incremental installation
<http://www.mercurylang.org/list-archives/reviews/2014-September/017263.html>

That's why I mentioned it specifically as a separate problem. :-)
Can the incremental installation be made to work with the (additive)
profiles proposed in the other thread?

In practice, we probably want to provide a separate tool (essentially a
package manager for grades) that handles the details -- the above diff
would just provide a mechanism to install the extra grades once the new
tool has worked out what new grades to install. So, yes, the new tool
could be aware of the installation profile.

It might go something like this:

./new_tool install-profile debug

which would do the following:

* query the compiler to see what grades are currently installed
* query the compiler for any other information about the current
installation (e.g. what the best LLDS base grade is)
* compute the difference between the set of currently installed grades
and those implied by the debug profile.
* if that difference is non-empty invoke configure with the
--install-extra-grades option and install the grades.

Julien.

[Mark Brown]

Post by Julien Fischer
In practice, we probably want to provide a separate tool (essentially a
package manager for grades) that handles the details -- the above diff
would just provide a mechanism to install the extra grades once the new
tool has worked out what new grades to install. So, yes, the new tool
could be aware of the installation profile.
./new_tool install-profile debug
* query the compiler to see what grades are currently installed
* query the compiler for any other information about the current
installation (e.g. what the best LLDS base grade is)
* compute the difference between the set of currently installed grades
and those implied by the debug profile.
* if that difference is non-empty invoke configure with the
--install-extra-grades option and install the grades.

Sounds good. Also you've answered my question from the other thread.

Cheers,
Mark.

[Paul Bone]

Post by Paul Bone
Is now a good time for a new Mercury release?

I've re-read through this list and made a list of the changes that people
want before we make a release. Only three directly referenced issues are
files as bugs is mantis.

https://www.mercurylang.org/bugs/roadmap_page.php

The others are here:

Zoltan's type status change

Zoltan's invalid data in the .d file bug

There's some more standard library issues that aren't currently in the bug
database, for example float.round_to_int/1 and friends have inconsistent
behaviours across different backends when their argument won't fit in an
int.

Zoltan's Grade selection proposal, if accepted.

Julien's Installation profiles (seems like this is accepted).

Duplicate output in .par grades on linux - pbone

Stack segments as default.

Anything else?

[Julien Fischer]

Hi Paul,

Post by Paul Bone

Post by Paul Bone
Is now a good time for a new Mercury release?

I've re-read through this list and made a list of the changes that people
want before we make a release. Only three directly referenced issues are
files as bugs is mantis.
https://www.mercurylang.org/bugs/roadmap_page.php

I've added some more, all of which are things that cause the compiler
to abort or generated executables to behave incorrectly. (And that
don't require the implementation of a new mode checker to fix!)

A few of the existing bugs have actually been fixed, I've marked them
as resolved and added regression tests where necessary.

I'm still going through the bug database, I'll post a more complete list
of things that (ideally) should be addressed before the release later.

Post by Paul Bone
Zoltan's type status change
Zoltan's invalid data in the .d file bug
There's some more standard library issues that aren't currently in the bug
database, for example float.round_to_int/1 and friends have inconsistent
behaviours across different backends when their argument won't fit in an
int.
Zoltan's Grade selection proposal, if accepted.

Ideally, yes. The changes to the grade component command line options
certainly need to happen, since we want to deprecate the meanings of
the existing options.

Post by Paul Bone
Julien's Installation profiles (seems like this is accepted).
Duplicate output in .par grades on linux - pbone

What's this one?

Post by Paul Bone
Stack segments as default.
Anything else?

Trail segments as default.

Installation of extra grades. In one of the recent threads, I proposed
we write a new tool to manage this -- I don't know whether this will be
done in time, but provided the changes to the configure script pass
review I see no reason not to commit them now.

There's a change from Mark, "variable bindings for interactive queries",
that I think also ought to go in.
Ref: <http://www.mercurylang.org/list-archives/reviews/2015-May/017956.html>

There's bunch of issues where program behaviour differs between
backends, for example, whether finalisers are run if main/2 terminates
with an uncaught exception.

Julien.

[Paul Bone]

Post by Julien Fischer
Hi Paul,

Post by Paul Bone

Post by Paul Bone
Is now a good time for a new Mercury release?

I've re-read through this list and made a list of the changes that people
want before we make a release. Only three directly referenced issues are
files as bugs is mantis.
https://www.mercurylang.org/bugs/roadmap_page.php

I've added some more, all of which are things that cause the compiler
to abort or generated executables to behave incorrectly. (And that
don't require the implementation of a new mode checker to fix!)
A few of the existing bugs have actually been fixed, I've marked them
as resolved and added regression tests where necessary.
I'm still going through the bug database, I'll post a more complete list
of things that (ideally) should be addressed before the release later.

Post by Paul Bone
Zoltan's type status change
Zoltan's invalid data in the .d file bug
There's some more standard library issues that aren't currently in the bug
database, for example float.round_to_int/1 and friends have inconsistent
behaviours across different backends when their argument won't fit in an
int.
Zoltan's Grade selection proposal, if accepted.

Ideally, yes. The changes to the grade component command line options
certainly need to happen, since we want to deprecate the meanings of
the existing options.

Post by Paul Bone
Julien's Installation profiles (seems like this is accepted).
Duplicate output in .par grades on linux - pbone

What's this one?

Something I noticed in some of the test suite. I'll reproduce it and file a
bug.

Post by Julien Fischer

Post by Paul Bone
Stack segments as default.
Anything else?

Trail segments as default.
Installation of extra grades. In one of the recent threads, I proposed
we write a new tool to manage this -- I don't know whether this will be
done in time, but provided the changes to the configure script pass
review I see no reason not to commit them now.
There's a change from Mark, "variable bindings for interactive queries",
that I think also ought to go in.
Ref: <http://www.mercurylang.org/list-archives/reviews/2015-May/017956.html>
There's bunch of issues where program behaviour differs between
backends, for example, whether finalisers are run if main/2 terminates
with an uncaught exception.

Like this one:

https://www.mercurylang.org/bugs/view.php?id=336

All of those seem reasonable to me.

[Julien Fischer]

Post by Paul Bone

Post by Julien Fischer
There's bunch of issues where program behaviour differs between
backends, for example, whether finalisers are run if main/2 terminates
with an uncaught exception.

https://www.mercurylang.org/bugs/view.php?id=336

Yes, that and #372.

Julien.