[m-dev.] status of --deep-profile-tail-recursion

[Julien Fischer]

Hi,

--deep-profile-tail-recursion is currently turned off by default.
There is a comment above that option (written by Paul in commit bda24d87)
that says:

% We should always handle tail recursion specially in deep
% profiling; the option is only for benchmarks for the paper,
% except that this is currently broken, and not supported with
% coverage profiling.

Was the brokenness mentioned in that comment fixed by the following change of
Zoltan's,
<http://www.mercurylang.org/list-archives/reviews/2011-July/015284.html>?

What is the status of --deep-profile-tail-recursion and --coverage-profiling?
Coverage profiling is enabled by default so if I enable
--deep-profile-tail-recursion do I need to disable --coverage-profiling?

Cheers,
Julien.

[Paul Bone]

Post by Julien Fischer
Hi,
--deep-profile-tail-recursion is currently turned off by default.
There is a comment above that option (written by Paul in commit bda24d87)
% We should always handle tail recursion specially in deep
% profiling; the option is only for benchmarks for the paper,
% except that this is currently broken, and not supported with
% coverage profiling.
Was the brokenness mentioned in that comment fixed by the following change of
Zoltan's,
<http://www.mercurylang.org/list-archives/reviews/2011-July/015284.html>?

I'm not sure if this fixes it either:
not at all,
partially (specific cases)
completely (general case)

I know that the borkenness pre-dates my work on deep profiling (before
2007).

Post by Julien Fischer
What is the status of --deep-profile-tail-recursion and --coverage-profiling?
Coverage profiling is enabled by default so if I enable
--deep-profile-tail-recursion do I need to disable --coverage-profiling?

That's a seperate issue and I don't remember the cause. Yes, it probably
makes these options mutually exclusive.

Generally when I cannot use tail recursion I either use a larger stack,
or stack segments. If both of those are unavailable I try to do something
like this:

foldl(P, Xs, !Acc) :-
foldl_2(P, 0, Xs, Rem, !Acc),
(
Rem = [_ | _],
foldl(P, Rem, !Acc)
;
Rem = []
).

foldl_2(_, _, [], [], !Acc).
foldl_2(P, Depth, [X | Xs], Rem, !Acc) :-
( Depth < 1000 ->
P(X, !Acc),
foldl_2(P, Depth + 1, Xs, !Acc)
;
Rem = [X | Xs]
).

You have to pick the right depth limit. In this example if I had a list of
1,000,000 items then rather than having 1,000,000 stack frames I have at
most only 1,000 + 1,000 frames.

[Zoltan Somogyi]

Post by Julien Fischer
Was the brokenness mentioned in that comment fixed by the following change of
Zoltan's,
<http://www.mercurylang.org/list-archives/reviews/2011-July/015284.html>?

Unfortunately, I do not remember.

Post by Julien Fischer
What is the status of --deep-profile-tail-recursion and --coverage-profiling?

The two were never designed to work together, so ...

Post by Julien Fischer
Coverage profiling is enabled by default so if I enable
--deep-profile-tail-recursion do I need to disable --coverage-profiling?

... the answer to this one is yes. If the two options happen to work together,
it would be by luck, not by design.

Zoltan.

[Julien Fischer]

Hi,

Post by Zoltan Somogyi

Post by Julien Fischer
Was the brokenness mentioned in that comment fixed by the following change of
Zoltan's,
<http://www.mercurylang.org/list-archives/reviews/2011-July/015284.html>?

Unfortunately, I do not remember.

Post by Julien Fischer
What is the status of --deep-profile-tail-recursion and --coverage-profiling?

The two were never designed to work together, so ...

The current situation where --deep-profile-tail-recursion is enabled by
default and --deep-profile-tail-recursion is not enabled by default is
not ideal, and also contrary to what the user's guide currently says,
namely:

With deep profiling, there are other modifications as well, the most
important impact of which is the loss of tail-recursion for groups of
mutually tail-recursive predicates (self-tail-recursive predicates stay
tail-recursive).

Assuming that --deep-profile-tail-recursion *is* working (I will look
into this as I happen to be profiling stuff at the moment anyway), are
there any objections to switching the default values of these two
options?

Cheers,
Julien.

[Zoltan Somogyi]

Post by Julien Fischer
The current situation where --deep-profile-tail-recursion is enabled by
default and --deep-profile-tail-recursion is not enabled by default is
not ideal, and also contrary to what the user's guide currently says,
With deep profiling, there are other modifications as well, the most
important impact of which is the loss of tail-recursion for groups of
mutually tail-recursive predicates (self-tail-recursive predicates stay
tail-recursive).
Assuming that --deep-profile-tail-recursion *is* working (I will look
into this as I happen to be profiling stuff at the moment anyway), are
there any objections to switching the default values of these two
options?

The thing is, deep profiling tail recursion is not as useful as it may appear,
which of course I found out the hard way. It has significant overheads,
and enabling it did not result in meaningful speedups. It does reduce stack
usage, but only for self-recursive predicates. Since it does not reduce
stack usage for mutually recursive predicates, you can still run out of
stack in a deep prof grade, and if I recall correctly, that used to happen
regularly to me when I forgot to set MERCURY_OPTIONS to avoid this.
(Several compiler tasks that operate on large data structures are
implemented using mutually recursive predicates.)

I think a better fix is to tie the grades that break tail call optimization
(which include the debug grades as well as the deep prof grades) to
the use of stack segments, and to fix the above quote in the user guide.

Zoltan.

[Julien Fischer]

Post by Zoltan Somogyi

Post by Julien Fischer
The current situation where --deep-profile-tail-recursion is enabled by
default and --deep-profile-tail-recursion is not enabled by default is
not ideal, and also contrary to what the user's guide currently says,
With deep profiling, there are other modifications as well, the most
important impact of which is the loss of tail-recursion for groups of
mutually tail-recursive predicates (self-tail-recursive predicates stay
tail-recursive).
Assuming that --deep-profile-tail-recursion *is* working (I will look
into this as I happen to be profiling stuff at the moment anyway), are
there any objections to switching the default values of these two
options?

The thing is, deep profiling tail recursion is not as useful as it may appear,
which of course I found out the hard way. It has significant overheads,
and enabling it did not result in meaningful speedups. It does reduce stack
usage, but only for self-recursive predicates. Since it does not reduce
stack usage for mutually recursive predicates, you can still run out of
stack in a deep prof grade, and if I recall correctly, that used to happen
regularly to me when I forgot to set MERCURY_OPTIONS to avoid this.
(Several compiler tasks that operate on large data structures are
implemented using mutually recursive predicates.)
I think a better fix is to tie the grades that break tail call optimization
(which include the debug grades as well as the deep prof grades) to
the use of stack segments, and to fix the above quote in the user guide.

The debug grades already use stack segments by default anyway. I will
adjust the configure script to make the profdeep grades do so as well.
(And update the user's guide and the comment in compiler/options.m
accordingly.)

Cheers,
Julien.