Status Update 2024 #1340
Replies: 3 comments 3 replies
-
Thanks @pointlessone, for everything you're doing! It maybe isn't said enough but I think most people using Prawn are very grateful that you invested time and effort under very difficult circumstances to get this latest release out! I certainly am. Being a fellow PDF library maintainer I can offer my few cents:
I would advocate that you stay in control of Prawn and related libraries for now. Even if you don't have much time, you have clearly shown that you care about Prawn and, if needed, there will be another, proper release. Maybe that's all that's needed for the time being. |
Beta Was this translation helpful? Give feedback.
-
Hi Alexander,
Thank you for this note, apologies for my delayed reply.
Thank you for your most recent release, it is a step in the right direction. Most Prawn users would agree our first priority is for Prawn to
have *"lights-on" maintenance*, i.e. be patched and released when dependency gems, external factors, and new Ruby versions cause breakages. Beyond that, enhancements and bug fixes are of course a nice-to-have.
It's hard for me to imagine the horrors you are living through in Ukraine. Given circumstances beyond your control--and anyone's control, besides perhaps that bastard in the Kremlin--I would again urge you to consider adding a trustworthy co-maintainer to the project. You noted that you have a lot of personal ego invested in Prawn, but perhaps it is time to let go for the greater benefit of the user community--it's the *responsible adult* thing to do.
Re: the fork I made, I've been pretty busy with work and travel these past months so I haven't had time to get it off the ground. My strong preference is for the *Prawn gem itself to be properly maintained*, and then no fork is necessary. If we cannot sort out the Prawn repo/gem maintainership situation within the next 6-12 months, then the fork will be a viable and legal option for the *community to move forward*.
In my case, I have 10,000 restaurant clients in Asia using Prawn to print out daily guest lists, and to generate e-receipt PDFs for guests--even
Disneyland is depending on Prawn! My own necessity for well-maintained Prawn (fork or otherwise) has everything to do with the happiness of my clients and engineering staff, and absolutely nothing do to with making a personal slight towards you, Alexander.
I wish you all the best in these difficult times. *Slava Ukraini!*
Best,
Johnny
|
Beta Was this translation helpful? Give feedback.
-
Thank you for your many years of effort and stewardship for PrawnPDF. As the last one standing on the core team and the person I trusted to carry the torch after my retirement, you've done an incredible job of keeping a very complex project alive, even if the demand for improvement and support greatly outpaces the means. My vote would be to keep the project alive as effectively "done software" with bug-fix only or easy improvements, and embrace (but not endorse) a fork for forward development if someone wants to go that route, or to see alternatives grow in capability, like HexaPDF, and any other alternatives. For me, I've always wanted to work on the problem of document building and only coincidentally learned what I had to to mostly stone soup this PDF library into existence, where most of the core team members (including yourself) brought the true expertise. This is a problem I believe I could solve well. But I don't think I could do by starting with an OSS project... instead I'd likely try to take on contracting gigs in various settings helping people who are using PrawnPDF or HexaPDF implement decent versions of what they're looking to build, filling in the gaps and using shoestrings and glue where needed to get a lot of real world experience with the range of needs people have. My best guess is to solve the general problem is borderline impossible, but to create the equivalent of a Tailwind for PDF is doable, and an equivalent of a Bootstrap, even moreso, and a BootstrapVue-like opinionated component framework, even moreso. At the far side of a good amount of work on such things, it could be extracted OSS, or a pay once model, or some combination thereof. Those seem like problems that are somewhat in their own category and in an ideal world, it'd make it possible for the underlying PDF engine to do less (whether it's PrawnPDF, a fork, or HexaPDF), in the domain of anything layout + document design related. As long as the raw tooling was there, the layout engine could essentially strip away all the layers of complexity and organically grown half-baked experiments. And this would likely reveal where the dividing line between these domains are. In other words, a new layout tool would ideally be built on top of the lowest level available APIs that are technically feasible to work with, and then work could be done to make sure those APIs were as stable as possible by patching up or replacing the core features in the underlying PDF library. What sucks is I always wanted to work on this side of things, but then spent the better part of a decade getting sucked down into the infinite morass that is PDF itself, which you, @gettalong, and so many others know far better than I do. I suppose to the extent that I can offer any help with this problem, I'd say... those who have clear, well-defined problems they simply want to get built out could reach out to me for contracting work ( gregory at practicingdeveloper.com ) -- and if I find a path towards some common tooling across different real world projects, I'll find some sort of way to make it more broadly available. And then of course, if I find bugs or rough edges in PrawnPDF's existing features along the path of building the things I need to, I'll start opening pull requests again and hopefully get some things fixed up. It's not much, but it's perhaps me signalling a desire to start stirring another stone soup, which could potentially put us in a better place by a few years from now. But I am old and tired now, so I can't quite be idealistic about things anymore. I am incredibly grateful to you all for keeping this project going, and for all the things that have come from it. It's a ridiculously difficult undertaking, and the fact we managed to build this project at all is a minor miracle. Thanks again. |
Beta Was this translation helpful? Give feedback.
-
Hi, long time no see!
You might've noticed there was a release recently.
I'm happy that I managed to do a bit of work on Prawn. I'm sure you all waited for it for a long time. It fixed a few bugs, and added a feature or two. But you might've noticed that it absolutely didn't fix all the bugs. So let's talk a bit about that.
You're holding it wrong
Well, not really. Those bugs are actual bugs but they're really hard to fix. In the process of investigating them I came to conclusion that the main reason it's hard to fix them is because most users have a wrong model of how Prawn works. Prawn is totally guilty of not making the model clear over the years but it seems people are coming with other models and just assume they would more or less apply to Prawn too. I don't blame them but this make the whole thing extra frustrating.
You see, Prawn is a streaming library. Pretty much anything you ask it to do it will immediately generate some bits of the final document. My understanding is that Prawn started like an experiment and it's extremely valuable to get something out fast and get some feedback immediately. This approach served well for the prototype and could produce good result in a variety of scenarios.
Early on, a few features were added that went slightly against this ethos. For example, if you look at the typical document that uses blocks to set bounding boxes and fonts, it starts looking like a completely different thing: HTML. We all know and love HTML, but there's one small issue here. HTML is declarative. HTML rendering happens in stages: first styles are resolves for all elements, then layout happens, and then everything is drawn. Nothing like this happens in Prawn. Here, as soon as we have a faintest idea we might just get away with painting, we do paint. As soon as we can fit a line of text, it's on the page.
Naturally, this streaming approach doesn't lend itself well to things that might need some planning.
Remember
group
? Everyone loves group. The problem is it can not reliably work in all circumstances. The main issue is it was rewinding a lot of state. Once it sees we're about to create a new page it would hack away all content generated so far, create a new page, and try again. You can imagine that this can result in a lot of grief and some corrupted documents.Same happens with other features that require hopping around the document. Repeaters, for example. It's very easy to come up with all sorts of weirdness like extra pages coming from seemingly nowhere, corrupted documents, and all that jazz.
So, what to do…
FLOSS (intermission 1)
Prawn is FLOSS. I mean, it's hard to actually classify. I don't know if the original Ruby license was officially approved by OSI but it's, at least in spirit, adjacent.
Now, have you ever noticed that no FLOSS license mentions maintainers? I mean, I haven't read every single license ever but the most popular ones don't. Likewise, there's very little acknowledgement of maintainers on official sites. OpenSource.org does somewhat better, but FSF has like one document for maintainers and it's mostly legal/GNU-specific information.
My current theory is that this is very intentional. Especially on the FSF's part. In their definition: ""Free software" means software that respects users' freedom and community. Roughly, it means that the users have the freedom to run, copy, distribute, study, change and improve the software." Basically, it's all about software. Whatever happens, software must persist. I guess that's a fine outlook to have when you are just starting the whole FOSS thing. One noticeable thing about this is that every user is sort of a maintainer of their own copy. They have all the same rights and responsibilities as all the other users. This too is somewhat understandable. It's kinda hard to advocate central development when patches have to be sent by air mail over the ocean.
In hindsight, it's kinda obvious why maintainers are a thing, though.
On one hand it's an acknowledgement of other's craftsmanship. In appreciation we honour authorship and so it happens that when we think of a piece of FLOSS we reflectively think of a few individuals even if in reality the actual software is the product of many-many people. For example, when I say Ruby on Rails do you think of a few thousands of contributors? Or the dozen people in the Core team? Or maybe even just DHH? I'm pretty confident it was not the first option. You might even be surprised to learn that over 6,600 contributes to Rails.
And on the other hand, it's an immense ego boost. Who wouldn't be proud of a ting that thousands of people across the whole world find useful? It certainly strokes my ego that I'm involved with such a popular project. I'm still pleased by the fact that Prawn is cited in a full score of scientific papers. I'm absolutely baffled by the fact that I'm named as one of the authors in one of those citations. I don't have a degree and my contribution to the whole scientific endeavour is totally inconsequential but I'm part of it because of Prawn.
There's an aspect of expertise, too. Some people have better knowledge of a particular piece of software. If only just because they spent a lot of time working on it. An average user wouldn't have a comparable level of knowledge. And so, people delegate to experts the task of fixing bugs and developing new features. Arguably, the result is better, but it, also arguably, introduces a failure mode that FSF and OSI don't want to think much about.
This notion of maintainer kinda throws a wrench in the whole software above all world view. The original conceit of Free Software seem to suggest that a more natural situation would be lots and lots of forks. Myriads of copies of software, each slightly different, evolving over time, borrowing from each other. Alas.
Speaking of which…
Fork!
Prawn has an official fork now! Johnny Shields took matters into his hands and forked Prawn.
I'll be honest, it was a little hit to my ego. But I can't complain, can I? I myself advocated for the fork. So I picked up pieces of what left of my ego and even tried to help make the fork good. I can't say Johnny heeded to all my advice but I can't blame him. I wasn't exactly… friendly with him. And certainly I wasn't a good maintainer to emulate.
The fork is at https://github.com/tablecheck/prawn-continued. You'll have to assess it for yourself, I'm afraid. I might have to declare a conflict of interest here.
Intermission (2)
Warning
Major bummer ahead.
The content provided "AS IS", without warranty of any kind, yada, yada…
Yes, it's a clickbait, and you know it and clicking on it despite being warned is totally on you.
I Accept
So let's talk about the war. What is it? You ask, which one? How terrible is it that that's even a question… We had the war to end all wars, and then another one, and we still keep starting new ones.
So let's talk about russia's war on Ukraine. I happen to live in Ukraine. More specifically I live on the north-west edge of Kyiv. Just a little over two years ago I had to leave my home to the sounds of gun fire and explosions.
For those of you who are not familiar with the local geography it's mere 10 km (a little over 6 freedom miles) from Bucha. You might've heard of it in the news, at some point it was big news even overseas.
It was February 25, 2022, the second day of war. It was extremely confusing. Everything was changing so fast and reliable information was scarce. Too many new experiences to process.
I was packing my stuff into car at the crack of dawn hearing rapid fire just on the other side of a small forest. I knew what I was hearing but it was hard to place it in the greater scheme of things. I newer fired a gun, I never saw one used in person. My only experience with guns was video games and movies. Intellectually I knew what I was hearing but emotionally it's very hard to reconcile that what is largely fiction for me suddenly spilled over into my reality.
I and my family spent a handful of months elsewhere in the country. It was a safe and peaceful place, given the circumstances. It gave me some space to come to terms with the new situation and at least try to adapt to it.
Eventually russians left the north of Ukraine. By the summer of 2022 we returned home because home is home.
In the autumn of 2022 russians started missile strikes. In a blink of an eye an intersection that I passed literally the day before became a crater. A playground in the park I walk often in is another crater.
They kept striking a couple times a week for a long time. Sometimes just at cities all over the country. Sometimes at electric grid infrastructure. They specifically targeted the grid. By December they managed to cause a few large blackouts.
At the end of November we had no power for over 3 days straight. I have to tell you that it's very inconvenient when most of your stuff is powered by electricity: all the usual stuff like phones, computers, and TVs, but also electric stove, boiler, lift (also known as freedom elevator) when you live on the floor a little higher than 12th. You also get reminded that a lot of stuff outside your home runs on electricity, too: like cell towers, internet, or even water pumps for running water and central heating. Closer to the end of it we only had cold canned food, candles, and books. It's a very strange experience to be in an over 3 million population city and be so isolated. You can read only so much by the candle. Mostly because actually you're waiting for the power to come back and get reconnected with the world, find out what's going on. Strange contemplative moods start to creep in.
Anyway, after that we had scheduled rotating blackouts. Everyone had to build their day around that schedule. 2 hours of power, maybe another 3 hours of power if electricians managed to patch the grid up enough, and then 4 hours without power. Every store had the schedule on the entrance. Every lift had the schedule on its doors so that people didn't get stuck. But if they did get stuck, every lift had a stool and a bag of snacks (like some cookies and a fruit) and a candle with matches. Quickly Perseverance centres were organised all over the country. There you could find warmth, power to change your stuff (they were equipped with a generator), water, internet, and first aid. Over the winter every business from a small kiosk to malls and big business centres had generators. And you could use any of those to charge your phones and power banks no questions asked. You could get basic necessities and use internet pretty much anywhere. It was heartwarming to see this level of unity.
But it was also very scary. At the time we didn't have good air defence. Every air strike meant there would be hits. This means that once you get an air strike alert you have to drop everything and take some precautions. At the very minimum you have to follow the rule of two walls: there must be at least two walls between you and a possible explosion. We all very quickly became familiar with structural properties of different types of buildings and how to identify the strongest structures like reinforced concrete sections of a wall.
Another thing we learned is that russian missiles are bit crap at aiming. Many of them were hitting seemingly empty space: right in the middle of an intersection, a playground in the park, some random field in the middle of nowhere, etc. On one hand it's kinda good because not much people actually died in these strikes, but on the other you can't be sure it won't hit your home. russians keep claiming that their missiles are high-precision weapon. But they kept hitting either nowhere or random civilian targets like malls and apartment blocks more often than any actual military targets. They seem to have built them crap on purpose to make them more effective weapons of terror.
By the summer of 2023 russians either realised that they can't fully destroy the grid or maybe just ran out of missiles and severely reduced intensity of air strikes. For just a little while it was relatively safe. It became warm and green outside and it was even possible to pretend for an hour or two that there was no war. But it didn't last much. russians laid their hands on some iranian drones. They started air strikes again.
By that time we had much better air defence. One factor is that we've got a bunch of modern systems from the partner states. Another is that our military got a lot of experience. We all got a lot of experience. We learned to distinguish by ear explosions of striking missiles, air defence missile launches, and controlled explosions of collected mines and leftover payloads of struck down missiles. I wish we didn't have to learn it but we have what we have.
We also learned a lot about the drones. These drones are basically slow cruise missiles with large wings. Because of that they can stay longer in the air and they need less space for manoeuvre. This makes it very hard to predict their target. They can literally fly in circles for hours. They also can fly much lower. A few times these drones flew at about the level of my floor right outside my window. Absolutely terrifying experience. 0/10, no amount of rise can save it, do not recommend. In comparison a cruise missile flies at at least twice the hight of my apartment tower and it's already far away before you can properly get scared. 2/10, still do not recommend.
Drones proved to be very unpredictable and very hard to shoot down. No one's going to use Patriot on them. One, it's too expensive, and two, it probably can't even be used because of how low the drones fly. The most effective way to deal with them is MANPADs (think, a shoulder rocket launcher) and big machine guns. The obvious problem is that you basically can't pulverise a drone this way, you can only shoot it down. And down it goes indeed. And down is where the people are. You can't chase the drone in the muddy field or forest. So the only option is along the roads and most roads are where people are. This inevitably means that if not a drone then at leas some big chunk of it might hit something. And we have to deal with this. Arguably it's better if an engine hits someone's flat than a drone hits and explodes a whole apartment block. It definitely sucks but one could make an argument that it's a lesser of two evils.
Do you remember the rule of two walls? Well, it's applicable here, too. Once you hear an air alert you have to take those precautions. One thing I haven't told you yet is that russians with their drones are basically orcs. They almost never launch drones by day, almost exclusively at night. Combined with the speed and distance there might be a considerable time before the drones reach my city. While normal people doomscroll Twitter or whatever, most people over here check air alerts map and Telegram channels to find out if there are any drones on the way.
Once in a while russians do combined attacks: drones and cruise missiles. Those missiles are launched from bombers. Those bombers are kept far away from Ukraine (all the way near Murmansk) to not get hit by our drones. The bombers also have to go from there to launch position over the Caspian Sea. Why there? Well, you see, as I said, russian missiles are a bit crap. A considerable percentage of them don't even fly. Obviously russians don't want to drop missiles with active payload over the most populated regions of their country. So they launch missiles over the sea. The defective ones just sink.
Anyway, those bombers need a couple of hours to reach the launch positions, then the missiles need a little bit of time to reach Ukraine, and then another half an hour to get near Kyiv. Like the blackout schedule this is an essential schedule for everyone here. Before going to sleep we check if there's any activity on the russian side. We know how much time everything might take and we plan our night accordingly. Sometimes it's easier to stay up a bit late. Sometimes you can get a few hours of sleep in your bed before you have to go to a bomb shelter or wherever those two walls are. You might take a nap in there for an hour or two. Then you might have another hour or a few of sleep in your bed or you might decide that today you're getting up early. You probably can imagine that this can't be good for one's health.
Now, why am I telling you all this?
First, I just need to get it out of my system. We, Ukrainians, talk about these experiences all the time, we understand each other and these shared experiences pretty well. But somehow it doesn't make it easier. I just hope that maybe, just maybe, sharing this with people who did not experience any of this might help me. Like releasing metaphorical gas of these experiences into an empty vessel of you would lessen the pressure in me just a bit at your expense. I'm sorry it has to be you. But also, you clicked "I accept".
Second, it's kind of an excuse for me being a meh maintainer. It's a weak excuse. I know I keep telling you all and myself all the time that I don't owe you anything, that it's all "AS IS", but it doesn't really work on me. It's still a responsibility, even if it's just implied. Even if it's self-imposed, because that's how I see a FOSS community. It still weighs on me that I couldn't meet my own expectations of myself.
I don't want you to leave with a false impression that I had it worse than anyone. To be honest, I didn't even had it particularly rough. If anything, I had a pretty mild experience compared to most Ukrainians. I'm relatively safe, all things considered. I live in comfort. None of my extended family and friends have died or even was injured so far. I wasn't directly involved in a military action. I contribute to the victory efforts in other ways but I shall not speak of it.
If you want a glimpse into alternative experience, read zverok's accounts (one, two, three, four, five). He lives in Kharkiv. His city was under occupation and is regularly shelled to this very day. Or Volodymyr Shymansky's, maintainer of wasm3 who's home was destroyed.
If you suddenly feel an urge for action you can write to your senator, MP, deputy, or other representative in your government and suggest them to more actively support Ukraine. You can learn more about the war and donate to support victory efforts. You can also donate to Come Back Alive Foundation.
Now, to get us back on track mood-wise here are pictures of my cats:
The Crossroads of Destiny
What lies ahead of us? What are the options?
Give people what they want
People clearly want a robust layout engine. Maybe not exactly HTML/CSS but something conceptually very close. It doesn't have to be as complex as modern CSS layout but it has to be a reliable layout engine. At the very least things like
group
has to come with the lib and work reliably every single time.Another thing people want is advanced text layout. Current Prawn can't even reliably layout all Western European scripts so the bar for "advanced" here is fairly low. But Ideally Unicode algorithms need to be implemented. We're used to fairly good text layout because major OSs and web browser are good at it. It's been probably years since you saw incorrectly rendered text last time. That is, unless you tried to do it in Prawn.
Unfortunately, I don't think this is happening. It will be a completely different lib. To the point that it's better to call it something else.
It's also going to be a huge project. I mean, PDF is large. Unicode algorithms are complex, too. And on top of that, a layout engine of arbitrary complexity?
You probably noticed that not all bugs are fixed in the last release. I tried. I looked into every single one. It certainly is possible to fix all bugs but it would require as much effort as a rewrite. And as much as my inner nerd is sniped and his neighbour idealistic craftsman called to action I have to tame the urge.
I have to admit it is an interesting project. I certainly am tempted once in a while to take it on. But let's be honest here, I can't invest that much effort into it. It's easily years worth of effort. And I'm, frankly speaking, too tired at the moment to even think about that much work. Also it'll have come with some sort of a business model attached to it. Because you lot are freeloaders.
Just kidding, I didn't let you pay me for Prawn. After a decade of maintaining it I still feel like it's mostly effort of all the other people. Both those that came before me and those that are contributing right now.
Anyway, I was saying that this is a hard to implement solution for many reasons. And definitely not a quick one, too. So overall, I'd say it's unlikely.
Keep on Keeping on
There's no denying that Prawn is useful. I've heard, it's mission-critical to at leas one fairly large business.
We can collectively try fixing bugs, and adding missing features, and improving things a little bit here and there, and be content with what we've got.
If I was too subtle, I am once again asking you for contributions. You don't need to be a maintainer to fix a bug, or write a bit of docs. Yes, the code is a mess. Yes, PDF/fonts/text are complex. But I'm not a superhuman either. You can learn and understand it. You don't have to understand all at once, too. Just that little corner that you want to fix/implement is often enough. Please scratch your itch.
Let it go
Let it go. Turn away and slam the door.
I mean… I am tired. I guess, you all are tired of me to some degree. Most of you probably are tired of how broken Prawn is.
And this whole maintainership take so much time and effort. It might not look like it but I was working nearly full time on Prawn gems since the end of October with a few weeks break around holidays. I was like two week in deep dive into fairly low level OpenType/CFF stuff after years of not much activity when Johnny announced his fork. Perfect timing.
There are alternatives out there.
HexaPDF seems nice. There are commercial solutions like PrincePDF, Antenna House Formatter, etc. There are external tools like Weasyprint. Even LaTeX and Typst can make you some PDFs.
I don't want to just abandon Prawn or give it away to whoever. If I were to transfer maintainership I'd like to make sure that that person has the expertise and is at least trying to do good for the whole community.
This is why I was so harsh with Johnny. I'm certain he has interest in Prawn but I'm convinced it's a particular type of interest and it doesn't align well with the broader community. But who knows, maybe his for will prove me wrong.
This is also why I was so reluctant to accept Peter's proposal to become a maintainer. I didn't know his expertise. He's quickly proven he's a diligent maintainer but I still was unsure whether he's familiar with PDF well enough. And then he went away. I'm sorry if it was my fault.
Vox Populi
I want to hear your opinion. Would you cast a vote for any of the options above if it was a poll? Do you have any other proposals? I really want to know. Please help me decide what to do next.
Thank you
In closing I want to thank you all. You made the last release possible. Some by directly contributing. Some by just being there, sitting in the bad corner of my imagination, judging me. :)
I want to specifically thank a few people:
Beta Was this translation helpful? Give feedback.
All reactions