<div dir="ltr"><div><div>A further use case would be to be able to convert all the locations to be relative, or include a relative portion, so that as tools manipulate the AST by adding or removing parts the layout can be preserved.<br>
<br></div>I think I may need to make a wip branch for this and experiment, it is always easier to comment on concrete things.<br><br></div>Alan<br></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Aug 28, 2014 at 10:38 PM, Simon Peyton Jones <span dir="ltr"><<a href="mailto:simonpj@microsoft.com" target="_blank">simonpj@microsoft.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div link="blue" vlink="purple" lang="EN-GB">
<div><div class="">
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:36.0pt">
I thiink the key question is whether it is acceptable to sprinkle this kind of information throughout the AST. For someone interested in source-to-source conversions (like me) this is great, others may find it intrusive.<u></u><u></u></p>
</div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">It’s probably not too bad if you use record syntax; thus<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> | HsDo { hsdo_do_loc :: SrcSpan -- of the word "do"<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> , hsdo_blocks :: BlockSrcSpans<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> , hsdo_ctxt :: HsStmtContext Name<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> , hsdo_stmts :: [ExprLStmt id]<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> , hsdo_type :: PostTcType }<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Simon<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div>
<div style="border:none;border-top:solid #e1e1e1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"" lang="EN-US">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif"" lang="EN-US"> Alan & Kim Zimmerman [mailto:<a href="mailto:alan.zimm@gmail.com" target="_blank">alan.zimm@gmail.com</a>]
<br>
<b>Sent:</b> 28 August 2014 19:35<br>
<b>To:</b> Richard Eisenberg<br>
<b>Cc:</b> Simon Peyton Jones; <a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
<b>Subject:</b> Re: GHC AST Annotations<u></u><u></u></span></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm">
This does have the advantage of being explicit. I modelled the initial proposal on HSE as a proven solution, and I think that they were trying to keep it non-invasive, to allow both an annotated and non-annoted AST.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm">
I thiink the key question is whether it is acceptable to sprinkle this kind of information throughout the AST. For someone interested in source-to-source conversions (like me) this is great, others may find it intrusive.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm">
The other question, which is probably orthogonal to this, is whether we want the annotation to be a parameter to the AST, which allows it to be overridden by various tools for various purposes, or fixed as in Richard's suggestion.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
A parameterised annotation allows the annotations to be manipulated via something like for HSE:<br>
<br>
-- |AST nodes are annotated, and this class allows manipulation of the annotations.<br>
class Functor ast => Annotated ast where<br>
<br>
-- |Retrieve the annotation of an AST node.<br>
ann :: ast l -> l<br>
<br>
-- |Change the annotation of an AST node. Note that only the annotation of the node itself is affected, and not<br>
-- the annotations of any child nodes. if all nodes in the AST tree are to be affected, use fmap.<u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
amap :: (l -> l) -> ast l -> ast l<u></u><u></u></p>
</div>
</div>
</div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
<u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
Alan<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm">
<u></u> <u></u></p>
<div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
On Thu, Aug 28, 2014 at 7:11 PM, Richard Eisenberg <<a href="mailto:eir@cis.upenn.edu" target="_blank">eir@cis.upenn.edu</a>> wrote:<u></u><u></u></p>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
For what it's worth, my thought is not to use SrcSpanInfo (which, to me, is the wrong way to slice the abstraction) but instead to add SrcSpan fields to the relevant nodes. For example:<br>
<br>
| HsDo SrcSpan -- of the word "do"<br>
BlockSrcSpans<br>
(HsStmtContext Name) -- The parameterisation is unimportant<br>
-- because in this context we never use<br>
-- the PatGuard or ParStmt variant<br>
[ExprLStmt id] -- "do":one or more stmts<br>
PostTcType -- Type of the whole expression<br>
<br>
...<br>
<br>
data BlockSrcSpans = LayoutBlock Int -- the parameter is the indentation level<br>
... -- stuff to track the appearance of any semicolons<br>
| BracesBlock ... -- stuff to track the braces and semicolons<br>
<br>
<br>
The way I understand it, the SrcSpanInfo proposal means that we would have lots of empty SrcSpanInfos, no? Most interior nodes don't need one, I think.<br>
<br>
Popping up a level, I do support the idea of including this info in the AST.<br>
<br>
Richard<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
<br>
On Aug 28, 2014, at 11:54 AM, Simon Peyton Jones <<a href="mailto:simonpj@microsoft.com" target="_blank">simonpj@microsoft.com</a>> wrote:<br>
<br>
> In general I’m fine with this direction of travel. Some specifics:<br>
><br>
> · You’d have to be careful to document, for every data constructor in HsSyn, what the association between the [SrcSpan] in the SrcSpanInfo and the “sub-entities”<br>
> · Many of the sub-entities will have their own SrcSpanInfo wrapped around them, so there’s some unhelpful duplication. Maybe you only want the SrcSpanInfo to list the [SrcSpan]s for the sub-entities (like the syntactic keywords) that do not show up
as children in the syntax tree?<br>
> Anyway do by all means create a GHC Trac wiki page to describe your proposed design, concretely.<br>
><br>
> Simon<br>
><br>
> From: ghc-devs [mailto:<a href="mailto:ghc-devs-bounces@haskell.org" target="_blank">ghc-devs-bounces@haskell.org</a>] On Behalf Of Alan & Kim Zimmerman<br>
> Sent: 28 August 2014 15:00<br>
> To: <a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
> Subject: GHC AST Annotations<br>
><br>
> Now that the landmines have hopefully been cleared from the AST via [1] I would like to propose changing the location information in the AST.<br>
><br>
> Right now the locations of syntactic markers such as do/let/where/in/of in the source are discarded from the AST, although they are retained in the rich token stream.<br>
><br>
> The haskell-src-exts package deals with this by means of using the SrcSpanInfo data type [2] which contains the SrcSpan as per the current GHC Located type but also has a list of SrcSpan s for the syntactic markers, depending on the particular AST fragment
being annotated.<br>
><br>
> In addition, the annotation type is provided as a parameter to the AST, so that it can be changed as required, see [3].<br>
><br>
> The motivation for this change is then<br>
><br>
> 1. Simplify the roundtripping and modification of source by explicitly capturing the missing location information for the syntactic markers.<br>
><br>
> 2. Allow the annotation to be a parameter so that it can be replaced with a different one in tools, for example HaRe would include the tokens for the AST fragment leaves.<br>
><br>
> 3. Aim for some level compatibility with haskell-src-exts so that tools developed for it could be easily ported to GHC, for example exactprint [4].<br>
><br>
><br>
><br>
> I would like feedback as to whether this would be acceptable, or if the same goals should be achieved a different way.<br>
><br>
><br>
><br>
> Regards<br>
><br>
> Alan<br>
><br>
><br>
><br>
><br>
> [1] <a href="https://phabricator.haskell.org/D157" target="_blank">https://phabricator.haskell.org/D157</a><br>
><br>
> [2] <a href="http://hackage.haskell.org/package/haskell-src-exts-1.15.0.1/docs/Language-Haskell-Exts-SrcLoc.html#t:SrcSpanInfo" target="_blank">
http://hackage.haskell.org/package/haskell-src-exts-1.15.0.1/docs/Language-Haskell-Exts-SrcLoc.html#t:SrcSpanInfo</a><br>
><br>
> [3] <a href="http://hackage.haskell.org/package/haskell-src-exts-1.15.0.1/docs/Language-Haskell-Exts-Annotated-Syntax.html#t:Annotated" target="_blank">
http://hackage.haskell.org/package/haskell-src-exts-1.15.0.1/docs/Language-Haskell-Exts-Annotated-Syntax.html#t:Annotated</a><br>
><br>
> [4] <a href="http://hackage.haskell.org/package/haskell-src-exts-1.15.0.1/docs/Language-Haskell-Exts-Annotated-ExactPrint.html#v:exactPrint" target="_blank">
http://hackage.haskell.org/package/haskell-src-exts-1.15.0.1/docs/Language-Haskell-Exts-Annotated-ExactPrint.html#v:exactPrint</a><br>
><u></u><u></u></p>
</div>
</div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:12.0pt;margin-left:0cm">
> _______________________________________________<br>
> ghc-devs mailing list<br>
> <a href="mailto:ghc-devs@haskell.org" target="_blank">ghc-devs@haskell.org</a><br>
> <a href="http://www.haskell.org/mailman/listinfo/ghc-devs" target="_blank">http://www.haskell.org/mailman/listinfo/ghc-devs</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal" style="margin-right:0cm;margin-bottom:6.0pt;margin-left:0cm">
<u></u> <u></u></p>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br></div>