[web-devel] Sql join feature in persistent

[Michael Snoyman]

I would imagine that Max is saying he'd like to see some built-in
functions that automate the forkIO/mvar/tvar code, the same way we are
discussing automating the join code. Am I right?

Michael

On Wed, Feb 23, 2011 at 4:25 PM, Greg Weber <greg at gregweber.info> wrote:
> What is stopping you from using forkIO and a mvar/tvar now?
> If all the use cases could be thought through, parallel queries might be
> able to be implemented transparently. But what if you don't want parallel
> queries- then we end up with a leaky abstraction where you expected a
> previous query to have already executed.
> On Tue, Feb 22, 2011 at 8:42 AM, Max Cantor <mxcantor at gmail.com> wrote:
>>
>> parallel queries would be helpful too.
>>
>> say i need to load independent data from 3 tables.  now that has 3
>> roundtrips of latency to the DB.  if there was a simple way to forkIO
>> PersistentBackend functions and dump the value in an mvar, it might help.
>>
>> would anyone else be interested in this?
>>
>> max
>>
>> On Feb 22, 2011, at 11:53 PM, Michael Snoyman wrote:
>>
>> > Max: Let's address your desire for sharding in a separate thread: I
>> > think it's a good idea, and it's a feature I would like to implement.
>> > I also think that your approach seems fairly feasible and not too
>> > disruptive. I want to make sure that introducing sharding code doesn't
>> > negatively affect users not taking advantage of it.
>> >
>> > As for joins (and other relational features): we should definitely add
>> > some helpers to Persistent. The main questions are:
>> >
>> > * What should their type signatures be?
>> > * Should we offer optimizations for the SQL backends to use proper
>> > JOINs?
>> >
>> > For the first, I think we still need to clarify a bit. Anton's code is
>> > a good start, but it certainly won't be amenable to SQL-backend
>> > optimizations. And I think we *should* strive to produce some
>> > efficient SQL code, though I want to avoid pouring too much effort
>> > into such an endeavor. Perhaps if we can write some code to handle the
>> > most common cases (eg, a 1-to-many relation between two tables) we'll
>> > be adhering well to the 80/80 rule.
>> >
>> > My ideal would be having some nice high-level functions that handle
>> > joins, optimize those to proper SQL joins where possible, and for
>> > backends where that is not possible simply do the join logic in
>> > Haskell.
>> >
>> > I definitely won't have time to get started on this coding for the
>> > next few weeks, but if anyone is interested in trying to implement any
>> > of the features under discussion, let me know, and I'll be happy to
>> > offer any assistance/code review I can.
>> >
>> > Michael
>> >
>> > On Tue, Feb 22, 2011 at 4:39 PM, Greg Weber <greg at gregweber.info> wrote:
>> >> The basic Persistent backend is designed without joins ("web-scale"),
>> >> and
>> >> with the idea that it should be SQL agnostic. App-level join
>> >> functionality
>> >> is definitely needed.
>> >> If someone wanted to design a layer on top of it with other SQL
>> >> features
>> >> that just the SQL backends can use that is fine. However, this might
>> >> start
>> >> re-inventing the wheel. For those interested in SQL joins and otherwise
>> >> advanced SQL queries, it would be great if they started trying out a
>> >> better
>> >> tool for the job, like the recently released DSH [1] or (bitrotting?)
>> >> HaskellDB. Relational Algebra sql query libraries can be very
>> >> productive to
>> >> use for 95% of use cases, even if they aren't "web-scale".
>> >> [1] http://hackage.haskell.org/package/DSH
>> >> [2] http://hackage.haskell.org/package/TableAlgebra
>> >> _______________________________________________
>> >> web-devel mailing list
>> >> web-devel at haskell.org
>> >> http://www.haskell.org/mailman/listinfo/web-devel
>> >>
>> >>
>> >
>> > _______________________________________________
>> > web-devel mailing list
>> > web-devel at haskell.org
>> > http://www.haskell.org/mailman/listinfo/web-devel
>>
>
>