<div dir="ltr">oh, i didn't realize you had already done the work! (bah, i'm sorry, i feel terrible)<div><br></div><div style>I thought i had communicated ~ a month ago that I was worried about release engineering interaction with making it impossible to then make a subsequent changes more thoughtfully because of the LLVM release cycle. This concern of mine balloned a bit after helping triage a huge number of problems people were hitting with the Clang transition on mac thats underway. </div>
<div style><br></div><div style>Its actually very easy to package up an llvm with that patch, much simpler than "build GHC from source". In fact, on OS X, the simplest way to install LLVM by default essentially does a build from source. </div>
<div style><br></div><div style>Geoff, it'd at least be worth running the benchmarks to measure the work! (and as I said, i'm happy to help)</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 12, 2013 at 2:30 PM, Geoffrey Mainland <span dir="ltr"><<a href="mailto:mainland@apeiron.net" target="_blank">mainland@apeiron.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">If users have to do a custom llvm build, we might as well ask them to<br>
build ghc from source too.<br>
<br>
Unless I misunderstood ticket #8033, you were originally quite gung-ho<br>
about changing the LLVM calling conventions to support passing SIMD<br>
vectors of all widths in registers on both x86-32 and -64, getting these<br>
patches into LLVM 3.4, and making sure that GHC 7.8 would support all<br>
this. I spent several days making sure this could happen from the GHC<br>
side. Now that the plan has changed, I will back out that work, and 7.8<br>
will only support passing 128-bit SIMD vectors in registers on x86-64.<br>
Other vectors sizes, and all vectors on x86-32, will be passed on the stack.<br>
<br>
Geoff<br>
<div class="im"><br>
On 9/12/13 1:32 PM, Carter Schonwald wrote:<br>
> to repeat:<br>
><br>
> I think no one would have object to having a clearly marked,<br>
> experimental -fllvmExpermentalAVX flag that requires building LLVM<br>
> with a specified patch, as a way to showcase your multivector work!<br>
><br>
> that would evade all of my objections (provided avx is still exposed<br>
> with normal -fllvm, but spilled to stack rather than registers), and<br>
> i'd actually argue in favor of such.<br>
><br>
> Especially since it would not impose any release cycle constraints on<br>
> a subsequent, systematic exploration for using XMM / YMM / ZMM in the<br>
> calling convention going forward.<br>
><br>
> @Geoff, Simons, Johan, and others: does anyone object to that approach?<br>
><br>
> applying such a calling convention patch to llvm is really quite<br>
> straightforward, and the build process is pretty zippy after that too.<br>
><br>
> cheers<br>
> -Carter<br>
><br>
><br>
> On Thu, Sep 12, 2013 at 2:34 AM, Carter Schonwald<br>
</div><div class="im">> <<a href="mailto:carter.schonwald@gmail.com">carter.schonwald@gmail.com</a> <mailto:<a href="mailto:carter.schonwald@gmail.com">carter.schonwald@gmail.com</a>>> wrote:<br>
><br>
> that said it does occur to me that there is an alternative<br>
> solution that may be acceptable for everyone!<br>
><br>
> what about providing a pseudo compatible way called<br>
> -fllvm-experimentalAVX (or something), and simply require that for<br>
> it to be used, the user has an llvm Patched with the YMM simd in<br>
> register fun call support? internally that could just be an llvm<br>
> way that trips the logic that puts the first few AVX values in<br>
> those YMM1-6 slots if they are the first args, so only the stack<br>
> spilling logic needs be changed?<br>
><br>
> (ie it wouldn't be tied to an llvm version, but rather this pseduo<br>
> way flag)<br>
><br>
> does that make sense?<br>
><br>
> either way, i'd really like having avx even if its always spilled<br>
> to stack at funcalls with standard LLVMs!<br>
><br>
> cheers<br>
> -carter<br>
><br>
><br>
><br>
><br>
> On Thu, Sep 12, 2013 at 2:28 AM, Carter Schonwald<br>
</div>> <<a href="mailto:carter.schonwald@gmail.com">carter.schonwald@gmail.com</a> <mailto:<a href="mailto:carter.schonwald@gmail.com">carter.schonwald@gmail.com</a>>><br>
<div class="HOEnZb"><div class="h5">> wrote:<br>
><br>
> Geoff,<br>
><br>
> a prosaic reason why there *might* be a fundamentally breaking<br>
> change would be the following idea nathan howell suggested to<br>
> me this afternoon: change the Sp and SPLim register so that<br>
> the X86/x86_64 target can use the CPU's Push and (maybe) Pop<br>
> instructions for the stack manipulations, rather than MOV and<br>
> fam. see <a href="http://ghc.haskell.org/trac/ghc/ticket/8272" target="_blank">http://ghc.haskell.org/trac/ghc/ticket/8272</a> (which<br>
> is just what i've said). Thats one change thats pretty simple<br>
> but deep, but likely worth exploring.<br>
><br>
><br>
> i'm saying any ABI change for GHC 7.10, would likely entail<br>
> patching LLVM 3.4, because thats the only LLVM version likely<br>
> to come out between now and whenever we get 7.10 out (assuming<br>
> 7.10 lands within the next 8-12 months, which is reasonable<br>
> since we've got noticeably more (amazing) people helping out<br>
> lately). Thus, any change there entails either asking the llvm<br>
> folks to support >1 GHC convention per architecture, or<br>
> replace the current one! I'd rather do the latter than the<br>
> former, when it comes to asking other people to maintain it :)<br>
> (and llvm engineers do in fact help out maintaining that code)<br>
><br>
><br>
> have you run a Nofib, or even benchmarks restricted to your<br>
> multivector code, for the current calling convention<br>
> (including the spilling AVX vectors to the stack thats the<br>
> current plan i gather) VS passing in registers with an LLVM<br>
> built using the patches i worked out ~ 2 months ago? it'd be<br>
> really easy to build that custom llvm, then run the<br>
> benchmarks! (i'm happy to help, and ultimately, benchmarks<br>
> will reveal if its worth while or not! And if the main goal is<br>
> for your talk, its still valid even if its not in the merge<br>
> window over the next 4 days).<br>
><br>
> I really think its not obvious what the "best" abi<br>
> change would be! It really will require coming up with a list<br>
> of variants, implementing them, and running nofib with each<br>
> variant, which i lack the compute/human time resources to do<br>
> this week. Modern hardware is complex enough that for<br>
> something like an ABI change, the only healthy attitude can be<br>
> "lets benchmark it!".<br>
><br>
> i'd really like any change in calling convention to also<br>
> improve perf on codes that aren't explicitly simd! (and a<br>
> conservative simd only change, blocks/conflicts with that<br>
> augmentation going forward, and not just for the stack pointer<br>
> example i mention early)<br>
><br>
> Not just scalar floats in simd registers , but perhaps also<br>
> words/ints !<br>
><br>
> (though that latter bit might be pretty ambitious and subtle,<br>
> i'll need to investigate that a bit to see how feasible it may<br>
> be).<br>
> SIMD has great support for ints/words, and any partial abi<br>
> change on the llvm backend now would make it hard to support<br>
> that later well (or at least, thats what it looks like to me).<br>
> actually effectively using simd for scalar ints and words<br>
> should be doable, but might force us to be a bit more<br>
> thoughtful on how GHC internally distinguishes ints used for<br>
> address arithmetic, vs ints used as data. (interestingly, i'm<br>
> not sure if any current extent x86 calling convention does that!)<br>
><br>
><br>
> That single change would make 7.10 require a completely<br>
> different llvm and native code gen convention from our current<br>
> one, plus touch all of the code gen on x86 architectures.<br>
><br>
><br>
> basically: we're lucky that everyone builds haskell code from<br>
> source, so ABI compat across GHC versions is a non issue. BUT,<br>
> any ABI changes should be backed by benchmarks (at least when<br>
> the change is performance motivated). Likewise, because we use<br>
> LLVM as an external dep for the -fllvm backend, we really need<br>
> to keep how their release cycle interacts with our release<br>
> cycle, because people use haskell and ghc! which as many like<br>
> to say, is both a boon and a pain ;).<br>
><br>
> Having people hit ghc acting broken with an llvm that was<br>
> "supported before" is risky support problem to deal with.<br>
> having an LLVM head variant support a modified ABI, and then<br>
> later needing to break it for 7.10 (for one of the possible<br>
> exploratory reasons above) would lead to a support headache I<br>
> don't wish on anyone.<br>
><br>
> pardon the verbose answer, but thats my offhand take<br>
><br>
> cheers<br>
> -Carter<br>
><br>
><br>
> On Wed, Sep 11, 2013 at 10:10 PM, Geoffrey Mainland<br>
</div></div><div class="im HOEnZb">> <<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>>> wrote:<br>
><br>
> We support compiling some code with -fllvm and some not in<br>
> the same<br>
> executable. Otherwise how could users of the Haskell<br>
> Platform link their<br>
> -fllvm-compiled code with native-codegen-compiled<br>
> libraries like base, etc.?<br>
><br>
> In other words, the LLVM and native back ends use the same<br>
> calling<br>
> convention. With my SIMD work, they still use the same calling<br>
> conventions, but the native codegen can never generate<br>
> code that uses<br>
> SIMD instructions.<br>
><br>
> Geoff<br>
><br>
> On 09/11/2013 10:03 PM, Johan Tibell wrote:<br>
> > OK. But that doesn't create a problem for the code we<br>
> output with the<br>
> > LLVM backend, no? Or do we support compiling some code<br>
> with -fllvm and<br>
> > some not in the same executable?<br>
> ><br>
> ><br>
> > On Wed, Sep 11, 2013 at 6:56 PM, Geoffrey Mainland<br>
> > <<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>><br>
</div><div class="HOEnZb"><div class="h5">> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>>>> wrote:<br>
> ><br>
> > We definitely have interop between the native<br>
> codegen and the LLVM<br>
> > back<br>
> > end now. Otherwise anyone who wanted to use the LLVM<br>
> back end<br>
> > would have<br>
> > to build GHC themselves. Interop means that users<br>
> can install the<br>
> > Haskell Platform and still use -fllvm when it makes<br>
> a performance<br>
> > difference.<br>
> ><br>
> > Geoff<br>
> ><br>
> > On 09/11/2013 07:59 PM, Johan Tibell wrote:<br>
> > > Do nothing different than you're doing for 7.8, we<br>
> can sort it out<br>
> > > later. Just put a comment on the primops saying<br>
> they're<br>
> > LLVM-only. See<br>
> > > e.g.<br>
> > ><br>
> > ><br>
> > ><br>
> ><br>
> <a href="https://github.com/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L181" target="_blank">https://github.com/ghc/ghc/blob/master/compiler/prelude/primops.txt.pp#L181</a><br>
> > ><br>
> > > for an example how to add docs to primops.<br>
> > ><br>
> > > I don't think we need interop between the native<br>
> and the LLVM<br>
> > > backends. We don't have that now do we (i.e. they<br>
> use different<br>
> > > calling conventions).<br>
> > ><br>
> > ><br>
> > ><br>
> > > On Wed, Sep 11, 2013 at 4:51 PM, Geoffrey Mainland<br>
> > > <<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>>><br>
> > <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>>>>> wrote:<br>
> > ><br>
> > > On 09/11/2013 07:44 PM, Johan Tibell wrote:<br>
> > > > On Wed, Sep 11, 2013 at 4:40 PM, Geoffrey<br>
> Mainland<br>
> > > <<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>>><br>
> > <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a><br>
> <mailto:<a href="mailto:mainland@apeiron.net">mainland@apeiron.net</a>>>>> wrote:<br>
> > > > > Do you mean we need a reasonable emulation<br>
> of the SIMD<br>
> > primops for<br>
> > > > > the native codegen?<br>
> > > ><br>
> > > > Yes. Reasonable in the sense that it<br>
> computes the right<br>
> > result.<br>
> > > I can<br>
> > > > see that some code might still want to<br>
> #ifdef (if the<br>
> > fallback isn't<br>
> > > > fast enough).<br>
> > ><br>
> > > Two implications of this requirement:<br>
> > ><br>
> > > 1) There will not be SIMD in 7.8. I just don't<br>
> have the<br>
> > time. In fact,<br>
> > > what SIMD support is there already will have<br>
> to be removed if we<br>
> > > cannot<br>
> > > live with LLVM-only SIMD primops.<br>
> > ><br>
> > > 2) If we also require interop between the LLVM<br>
> back-end and<br>
> > the native<br>
> > > codegen, then we cannot pass any SIMD vectors in<br>
> > registers---they all<br>
> > > must be passed on the stack.<br>
> > ><br>
> > > My plan, as discussed with Simon PJ, is to not<br>
> support SIMD<br>
> > primops at<br>
> > > all with the native codegen. If there is a<br>
> strong feeling that<br>
> > > this *is<br>
> > > not* the way to go, the I need to know ASAP.<br>
> > ><br>
> > > Geoff<br>
> > ><br>
> > ><br>
> > ><br>
> ><br>
> ><br>
><br>
><br>
><br>
><br>
<br>
</div></div></blockquote></div><br></div>