`pipes` is a group of libraries written in Haskell to provide safe, functional, and strict I/O.
Questions tagged [haskell-pipes]
135 questions
49
votes
3 answers
What is pipes/conduit trying to solve
I have seen people recommending pipes/conduit library for various lazy IO related tasks. What problem do these libraries solve exactly?
Also, when I try to use some hackage related libraries, it is highly likely there are three different versions.…
Sibi
- 43,989
- 14
- 80
- 146
25
votes
2 answers
Haskell fast concurrent queue
The Problem
Hello! I'm writing a logging library and I would love to create a logger, that would run in separate thread, while all applications threads would just send messages to it. I want to find the most performant solution for this problem. I…
Wojciech Danilo
- 10,776
- 13
- 60
- 118
19
votes
2 answers
What is the connection between Iteratees and FRP?
It seems to me that there is a strong connection between the two ideas. My guess is that FRP could be implemented in terms of Iteratees if there would be a way to express arbitrary graphs with Iteratees. But afaik they only support chain-like…
fho
- 6,578
- 19
- 63
14
votes
1 answer
If MonadPlus is the "generator" class, then what is the "consumer" class?
A Pipe can be broken into two parts: the generator part (yield) and the consumer part (await).
If you have a Pipe that only uses it's generator half, and only returns () (or never returns), then it can be represented as a "ListT done right". It…
Dan Burton
- 51,332
- 25
- 109
- 190
14
votes
3 answers
Idiomatic bidirectional Pipes with downstream state without loss
Say I have simple producer/consumer model where the consumer wants to pass back some state to the producer. For instance, let the downstream-flowing objects be objects we want to write to a file and the upstream objects be some token representing…
bgamari
- 5,265
- 1
- 16
- 21
14
votes
2 answers
What's the benefit of conduit's leftovers?
I'm trying to understand the differences between conduit and pipes. Unlike pipes, conduit has the concept of leftovers. What are leftovers useful for? I'd like to see some examples where leftovers are essential.
And since pipes don't have the…
Petr
- 60,177
- 8
- 136
- 295
13
votes
1 answer
pipes 3.0 : non linear topologies
I am having a look at the pipes 3.0 package for stream processing. The tutorial is very well done and very clear, except that I cannot wrap my head around the "zip and merge" section.
My goal is to combine pipes a bit like ArrowChoice allows to…
LeMiz
- 5,068
- 5
- 25
- 23
10
votes
1 answer
How can I idiomatically and efficiently consume a Pipe in some non-IO monad, with an IO action?
I have a Producer that creates values that depend on randomness, using my own Random monad:
policies :: Producer (Policy s a) Random x
Random is a wrapper over mwc-random that can be run from ST or IO:
newtype Random a =
Random (forall m.…
Tikhon Jelvis
- 64,915
- 16
- 168
- 210
10
votes
2 answers
Space leak in Pipes with RWST
A memory analysis of the following program shows that the noleak functions runs in constant memory while the leak function leaks memory in a linear fashion. dflemstr indicated that this might be due to RWST causing an infinite chain of allocations.…
prinsen
- 678
- 4
- 15
10
votes
2 answers
Haskell Pipes and Branching
Problem
I'm attempting to implement a simple web server with Haskell and the Pipes library. I understand now that cyclic or diamond topologies aren't possible with pipes, however I thought that what I am trying to is. My desired topology is thus:
…
Dwilson
- 1,219
- 9
- 18
10
votes
2 answers
What's the real benefit of conduit's upstream type parameter?
I'm trying to understand the differences between different implementations of the concept of pipes. One of the differences between conduit and pipes is how they fuse pipes together. Conduit has
(>+>) :: Monad m
=> Pipe l a b r0 m r1 -> Pipe…
Petr
- 60,177
- 8
- 136
- 295
9
votes
1 answer
Streaming parsing of JSON in Haskell with Pipes.Aeson
The Pipes.Aeson library exposes the following function:
decode :: (Monad m, ToJSON a) => Parser ByteString m (Either DecodingError a)
If I use evalStateT with this parser and a file handle as an argument, a single JSON object is read from the file…
immutablestate
- 287
- 1
- 8
8
votes
2 answers
Why does Haskell Pipes "use () to close unused inputs and X (the uninhabited type) to close unused outputs"?
In the Pipes Tutorial, it says that:
The concrete type synonyms use () to close unused inputs and X (the
uninhabited type) to close unused outputs:
I'd like to understand why () and X are used the way they are. Why not X or () for both inputs…
Ana
- 11,726
- 5
- 49
- 101
8
votes
1 answer
How to detect last chunk in a Haskell Pipe?
I have a small Haskell Pipe that prints out how many times it has run:
counterPipe :: Pipe String String IO r
counterPipe = go 0
where
go n = do
await >>= yield
let n' = succ n
liftIO $ putStrLn $ "Chunk " ++ show n'
go…
Ana
- 11,726
- 5
- 49
- 101
8
votes
2 answers
Pipes.Binary.decode - what is the StateT for?
I'm trying to write a basic network server using pipes and the assorted libraries that build on it. The intended flow would be:
get bytestring from socket -> decode using binary -> server logic goes here -> send response to socket
Which I figured…
user3261399
- 347
- 1
- 4