Mailing List Archive: Gentoo: Desktop

Interest inquery: kde4-nosemantic overlay

Index | Next | Previous | View Threaded

1i5t5.duncan at cox

Jul 3, 2013, 10:32 AM

Post #1 of 25 (260 views)
Permalink

Interest inquery: kde4-nosemantic overlay

For kde-4.11, it seems the gentoo/kde project has decided to hard-enable
the former semantic-desktop USE flag, forcing the option on and forcing a
number of formerly optional additional dependencies.[1]

But, I spent quite some time here switching away from kdepim's kmail,
akregator, etc, so I could kill akonadi on my system, and with it
semantic-desktop, etc, and I'm in no mood to have it hard-enabled now.
If it comes to it, I'd rather dump the kde desktop and switch to something
else[2], than have semantic-desktop on my system once again.

But with a bit of luck, I won't have to switch away from kde after all.

I already asked gentoo/kde to reconsider, given that they've supported
USE=-semantic-desktop until now and with 4.11 much of kde4's going into
maintenance mode as the upstream developer focus switches to kde5/kde-
frameworks, so it makes little sense to drop support for -semantic-
desktop now, when upstream is continuing to offer that option at least
thru kde4, and kde5/frameworks is supposed to be far more modular, so with
luck will allow users to pick and choose whether they want the
semantic-desktop components pulled in or not. However, given the gentoo/
kde project history with dropping kde3 support and forcing kde3 users to
to the user-supported kde-sunset overlay even while kde4 was still not
ready for use (and despite upstream kde's broken promise to support kde3
as long as there continued to be users), I'm not optimistic, but it was
worth a shot.

But the kde-sunset overlay does suggest another alternative, a kde4-
nosemantic overlay.

Meanwhile, as I upgraded to the kde-4.11 pre-releases (currently 4.10.90
aka 4.11-beta2) in the kde overlay, for the kde-desktop-core and other
gentoo/kde packages I still run, I diffed the ebuilds between 4.10.x and
4.10.80 (aka 4.11-beta1), then checked the diffs for non-semantic-desktop
related changes and kept them, while changing the semantic-desktop force-
enabling changes to force-disabling instead.

Then I created a framework that works much like epatch_user, except
instead of automatically applying patches to upstream sources, it
automatically applies patches to gentoo ebuilds and instead of using the
/etc/portage/patches/ tree, it uses /etc/portage/patches.ebuild/.

So now I have a set of ebuild patches that patch the kde 4.11 ebuilds
(starting with 4.10.80, aka 4.11-beta1) to force-disable semantic-
desktop, instead of force-enabling it. And I have a scripted framework
that auto-applies these patches to new ebuilds on emerge --sync and layman
-S, thus keeping no-semantic around as upstream gentoo/kde updates their
ebuilds.

For now, therefore, I'm fine, up and running on 4.10.90 (aka 4.11-beta2),
using gentoo/kde ebuilds auto-patched to kill the now forced-on semantic-
desktop, forcing it off instead.

But realistically, I honestly don't know if longer term, I'll be able to
continue maintenance of all of this by myself. Chances are unfortunately
high that without help from others, over time I'll decide it's simply too
much of a hassle maintaining the patches, and will end up switching to
some other desktop, with the qt-based razor-qt desktop one candidate as
sort of a kde-lite desktop, and enlightenment as another, getting away
from kde and qt entirely.

Besides which, if I'm finding kde-nosemantic useful enough to go to all
this trouble, there's a good chance that others will be interested in it
themselves, especially if they don't have to do all the work I'm now doing
myself, themselves. So with kde-sunset in mind as precedent, I'm now
proposing a kde-nosemantic overlay, like kde-sunset, user-maintained, but
for kde4 folks who want a continued no-semantic choice, instead of kde3
users.

Any interest?

To be further discussed: Assuming a go-ahead on the general idea, do we
want to maintain it as a normal overlay carrying at least the kde4 ebuilds
that require patching to kill semantic-desktop, or should we simply build
on the epatch_ebuild_user scripts I have hacked up, presumably checking
them into a git repo along with the patches themselves somewhere and
making that available, then simply use that tool with the existing gentoo
tree (when 4.11 is released and ebuilds arrive in the main tree) and kde
project overlay to apply the patches to the existing tree and overlay
instead of creating a full-fledged kde-nosemantic overlay ourselves. Of
course the tools and patches could then have ebuilds and appear in an
overlay of their own, rather than having the modified kde-nosemantic
ebuilds in an overlay.

One bonus to the tools overlay instead of a direct kde-nosemantic overlay
approach, is that gentooers not interested in kde, but interested in the
ebuild-patch tools, might find that useful, add that overlay to their
layman overlay list, and contribute patches to the ebuild-patches tool,
helping it mature and grow into a general purpose automated-ebuild-
patching tool rather faster than it might otherwise happen.

A hybrid alternative would be to adopt an idea much like the existing kde
overlay, where there's a documentation or tools directory that carries
them, in addition to the kde-base category and etc, carrying the patched
ebuilds themselves.

So what do people think? Any interest? How should we go about it?

Or should I just continue working on it on my own, with the likelihood
that at some point I'll decide it's not worth the trouble and switch to a
non-kde desktop, as I've switched to other non-kde tools as the kde
versions jumped the shark over the course of kde4?

In particular, I expect users who are or have been active in the kde-
sunset overlay will have some useful insights.

---
[1] Andreas Huttel, aka gentoo dev dilfridge, covered this on his blog
(which is in turn covered by planet-gentoo, where I subscribe to the feed,
thus seeing it there):

http://dilfridge.blogspot.com/2013/05/news-from-201305-gentoo-kde-team-meeting.html

[2] While during the early kde4 fiasco I was mostly standardized on kde
apps and therefore had little choice, over the course of kde4, I've
switched away from kde apps for first one thing than another, so by now
it's mostly the core kde4 desktop I depend on, plus a few other less vital
apps, games, dolphin, gwenview, superkaramba, that I could leave behind
far more easily now, if I decided I could no longer run the kde-
core-desktop.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

thev00d00 at gentoo

Jul 3, 2013, 11:48 PM

Post #2 of 25 (253 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

To be honest I see this as a huge over reaction.

Its unclear from your mail if you did try running it with it disabled at
runtime first before hacking the ebuilds... If you did not I do recommended
it just to see how little difference it actually makes.

I guess having all the hacks centralised will be useful at least though.

Ian
On 3 Jul 2013 18:33, "Duncan" <1i5t5.duncan [at] cox> wrote:

> For kde-4.11, it seems the gentoo/kde project has decided to hard-enable
> the former semantic-desktop USE flag, forcing the option on and forcing a
> number of formerly optional additional dependencies.[1]
>
> But, I spent quite some time here switching away from kdepim's kmail,
> akregator, etc, so I could kill akonadi on my system, and with it
> semantic-desktop, etc, and I'm in no mood to have it hard-enabled now.
> If it comes to it, I'd rather dump the kde desktop and switch to something
> else[2], than have semantic-desktop on my system once again.
>
> But with a bit of luck, I won't have to switch away from kde after all.
>
> I already asked gentoo/kde to reconsider, given that they've supported
> USE=-semantic-desktop until now and with 4.11 much of kde4's going into
> maintenance mode as the upstream developer focus switches to kde5/kde-
> frameworks, so it makes little sense to drop support for -semantic-
> desktop now, when upstream is continuing to offer that option at least
> thru kde4, and kde5/frameworks is supposed to be far more modular, so with
> luck will allow users to pick and choose whether they want the
> semantic-desktop components pulled in or not. However, given the gentoo/
> kde project history with dropping kde3 support and forcing kde3 users to
> to the user-supported kde-sunset overlay even while kde4 was still not
> ready for use (and despite upstream kde's broken promise to support kde3
> as long as there continued to be users), I'm not optimistic, but it was
> worth a shot.
>
> But the kde-sunset overlay does suggest another alternative, a kde4-
> nosemantic overlay.
>
> Meanwhile, as I upgraded to the kde-4.11 pre-releases (currently 4.10.90
> aka 4.11-beta2) in the kde overlay, for the kde-desktop-core and other
> gentoo/kde packages I still run, I diffed the ebuilds between 4.10.x and
> 4.10.80 (aka 4.11-beta1), then checked the diffs for non-semantic-desktop
> related changes and kept them, while changing the semantic-desktop force-
> enabling changes to force-disabling instead.
>
> Then I created a framework that works much like epatch_user, except
> instead of automatically applying patches to upstream sources, it
> automatically applies patches to gentoo ebuilds and instead of using the
> /etc/portage/patches/ tree, it uses /etc/portage/patches.ebuild/.
>
> So now I have a set of ebuild patches that patch the kde 4.11 ebuilds
> (starting with 4.10.80, aka 4.11-beta1) to force-disable semantic-
> desktop, instead of force-enabling it. And I have a scripted framework
> that auto-applies these patches to new ebuilds on emerge --sync and layman
> -S, thus keeping no-semantic around as upstream gentoo/kde updates their
> ebuilds.
>
> For now, therefore, I'm fine, up and running on 4.10.90 (aka 4.11-beta2),
> using gentoo/kde ebuilds auto-patched to kill the now forced-on semantic-
> desktop, forcing it off instead.
>
> But realistically, I honestly don't know if longer term, I'll be able to
> continue maintenance of all of this by myself. Chances are unfortunately
> high that without help from others, over time I'll decide it's simply too
> much of a hassle maintaining the patches, and will end up switching to
> some other desktop, with the qt-based razor-qt desktop one candidate as
> sort of a kde-lite desktop, and enlightenment as another, getting away
> from kde and qt entirely.
>
> Besides which, if I'm finding kde-nosemantic useful enough to go to all
> this trouble, there's a good chance that others will be interested in it
> themselves, especially if they don't have to do all the work I'm now doing
> myself, themselves. So with kde-sunset in mind as precedent, I'm now
> proposing a kde-nosemantic overlay, like kde-sunset, user-maintained, but
> for kde4 folks who want a continued no-semantic choice, instead of kde3
> users.
>
> Any interest?
>
> To be further discussed: Assuming a go-ahead on the general idea, do we
> want to maintain it as a normal overlay carrying at least the kde4 ebuilds
> that require patching to kill semantic-desktop, or should we simply build
> on the epatch_ebuild_user scripts I have hacked up, presumably checking
> them into a git repo along with the patches themselves somewhere and
> making that available, then simply use that tool with the existing gentoo
> tree (when 4.11 is released and ebuilds arrive in the main tree) and kde
> project overlay to apply the patches to the existing tree and overlay
> instead of creating a full-fledged kde-nosemantic overlay ourselves. Of
> course the tools and patches could then have ebuilds and appear in an
> overlay of their own, rather than having the modified kde-nosemantic
> ebuilds in an overlay.
>
> One bonus to the tools overlay instead of a direct kde-nosemantic overlay
> approach, is that gentooers not interested in kde, but interested in the
> ebuild-patch tools, might find that useful, add that overlay to their
> layman overlay list, and contribute patches to the ebuild-patches tool,
> helping it mature and grow into a general purpose automated-ebuild-
> patching tool rather faster than it might otherwise happen.
>
> A hybrid alternative would be to adopt an idea much like the existing kde
> overlay, where there's a documentation or tools directory that carries
> them, in addition to the kde-base category and etc, carrying the patched
> ebuilds themselves.
>
> So what do people think? Any interest? How should we go about it?
>
> Or should I just continue working on it on my own, with the likelihood
> that at some point I'll decide it's not worth the trouble and switch to a
> non-kde desktop, as I've switched to other non-kde tools as the kde
> versions jumped the shark over the course of kde4?
>
> In particular, I expect users who are or have been active in the kde-
> sunset overlay will have some useful insights.
>
> ---
> [1] Andreas Huttel, aka gentoo dev dilfridge, covered this on his blog
> (which is in turn covered by planet-gentoo, where I subscribe to the feed,
> thus seeing it there):
>
>
> http://dilfridge.blogspot.com/2013/05/news-from-201305-gentoo-kde-team-meeting.html
>
> [2] While during the early kde4 fiasco I was mostly standardized on kde
> apps and therefore had little choice, over the course of kde4, I've
> switched away from kde apps for first one thing than another, so by now
> it's mostly the core kde4 desktop I depend on, plus a few other less vital
> apps, games, dolphin, gwenview, superkaramba, that I could leave behind
> far more easily now, if I decided I could no longer run the kde-
> core-desktop.
>
> --
> Duncan - List replies preferred. No HTML msgs.
> "Every nonfree program has a lord, a master --
> and if you use the program, he is your master." Richard Stallman
>
>
>

1i5t5.duncan at cox

Jul 4, 2013, 3:15 AM

Post #3 of 25 (248 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Ian Whyman posted on Thu, 04 Jul 2013 07:48:01 +0100 as excerpted:

> To be honest I see this as a huge over reaction.
>
> Its unclear from your mail if you did try running it with it disabled
> at runtime first before hacking the ebuilds... If you did not I do
> recommended it just to see how little difference it actually makes.
>
> I guess having all the hacks centralised will be useful at least though.

FWIW, I did run it up thru kde 4.6. That's when I decided I didn't use
it anyway, so I might as well turn it off at build-time and avoid the
additional dependencies.

Now they /say/ semantic-desktop is far faster now, and easily run-time
disabled as well, and while I've certainly seen "the new, improved
semantic-desktop" story from kde enough times to not put a lot of
credence in that (not that kde was lying about the claims, but there's
"improved" and there's 'improved'...), the gentoo/kde devs do have
somewhat better credibility IMO and /they/ are saying it now, so I won't
argue that it's not the case.

So I'm not arguing that it can't be runtime disabled, or even that
performance might not have improved to the point where having it on isn't
a big deal either.

However, that doesn't change the fact that if I don't use it, I don't use
it, and I don't want or need all those additional deps (when I disabled
it, I was actually rather surprised at the number of packages I no longer
needed and was able to remove... only to allow them back on my system if
gentoo/kde had their way... which in this case they won't, not on my
system) it pulls in on my system, period. Among other things, having
unused stuff on one's system is bad security practice. One thing I've
found by experience about gentoo, and as it happens, generally like, is
that the very fact that because one is actually building all updates, not
simply installing binaries, over time and multiple updates it tends to
reasonably strongly discourage even having unused stuff installed -- thus
encouraging what's good security practice in any case. =:^) And of
course in the normal case, either simply removing the unused package from
the world file or tweaking USE flags to avoid pulling it in (plus the
depclean to actually remove it), is all that's necessary to remove it, if
indeed it's truly unused.

Which is what's so frustrating here, as gentoo/kde is subverting the
normal gentoo process and way, forcing entirely unneeded and unused
dependencies, unless users take drastic measures like creating the
necessary patches to revert the force, and a script to auto-apply said
patches them as updates occur.

As one of the comments on the previously linked blog post stated, Larry
the Cow isn't amused! =:^(

So yes, arguably it /is/ making a big deal out of nothing. OTOH, that's
what all the binary-distro folks say about the whole admin controls
optional deps via USE flags and actually building the package themselves,
too. If I didn't feel strongly about that sort of thing, why would I
even bother with all build-from-sources hassle of gentoo in the first
place? There's a number of reasons I'm a gentooer, and /none/ of them
are because I want distro package maintainers making my choices for me
about optional deps.

What's even more galling is that thru all of kde4 until 4.11, for a
number of kde packages declared the last feature release of the kde4
series, gentoo/kde has had the semantic-desktop USE flag. With kde5/kde-
frameworks, upstream kde is going far more modular, with a much smaller
core (for two reasons, as part of the base functionality is actually
moving to qt5 as well as the actual lower kde base package count, so the
kde core should be MUCH smaller indeed), making everything else
optional. Now I don't know for /sure/ that semantic-desktop is one (or
more) of the optional modules not part of core, but given it was optional
in kde4 and they're going more modular in kde5/frameworks, /not/ making
it optional in the latter would be a direct step backward from the
declared goal, so it should be reasonably unlikely.

Which means all of kde4 thru 4.10 would have had optional semantic-
desktop, and kde5/frameworks should have it optional as well, making hard-
enabling semantic-desktop for the 4.11 longer-term maintenance release
even less reasonable than it'd be if kde5 was going to force it on
upstream.

But as is so often said, in FLOSS, most devs are to a large extent simply
scratching their own itches, and it seems all the gentoo/kde devs use
semantic-desktop so dealing with the option is simply a hassle, not an
itch any of them has to scratch. Of course it was the same thing with
kde3/kde4, none of the gentoo/kde devs were interested in maintaining
kde3 any longer despite the fact that kde4 was still very broken for the
needs of many users, so they simply dropped it, or rather, pushed it off
into the user-maintained kde-sunset overlay. So I don't really expect
any different here, nor in fact am I really demanding it, tho it would
certainly be nice. I'm simply disappointed, is all. I'm prepared to do
what I must, but I'm disappointed, if not entirely surprised, that it
came to this. Oh, well...

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

wired at gentoo

Jul 4, 2013, 3:02 PM

Post #4 of 25 (247 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

On Wed, Jul 03, 2013 at 05:32:25PM +0000, Duncan wrote:
> I already asked gentoo/kde to reconsider, given that they've supported
> USE=-semantic-desktop until now and with 4.11 much of kde4's going into
> maintenance mode as the upstream developer focus switches to kde5/kde-
> frameworks, so it makes little sense to drop support for -semantic-
> desktop now, when upstream is continuing to offer that option at least
> thru kde4, and kde5/frameworks is supposed to be far more modular, so with
> luck will allow users to pick and choose whether they want the
> semantic-desktop components pulled in or not. However, given the gentoo/
> kde project history with dropping kde3 support and forcing kde3 users to
> to the user-supported kde-sunset overlay even while kde4 was still not
> ready for use (and despite upstream kde's broken promise to support kde3
> as long as there continued to be users), I'm not optimistic, but it was
> worth a shot.

Being part of the team that decided to remove KDE3 from the tree, I
assure you that it wasn't an easy decision. Unfortunately, KDE3 was in
really bad shape. Upstream had abandoned it, build tools slowly became
incompatible with it and maintaining it became a big PITA for the KDE
team.

A lot of work had been put into making KDE3 and KDE4 co-exist already,
since we too felt that KDE4 was not ready and I'm pretty sure we kept it
around longer than most distros out there.

But at some point KDE4 became usable and we had to move on, for our own
sanity.

--

That said, I do agree that semantic-desktop should stay optional in
KDE4. Back in my KDE days I hated it and always ensured it - and its
ugly dependencies - stayed off my system. Unfortunately I am not part of
the KDE team anymore, so I don't know what their reasoning is behind
this decision. Hopefully they will reconsider :)

--
Alex Alexander | wired [at] gento
+ www.linuxized.com
++ www.leetworks.com

1i5t5.duncan at cox

Jul 4, 2013, 8:22 PM

Post #5 of 25 (246 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Alex Alexander posted on Fri, 05 Jul 2013 01:02:24 +0300 as excerpted:

> Being part of the team that decided to remove KDE3 from the tree, I
> assure you that it wasn't an easy decision. Unfortunately, KDE3 was in
> really bad shape. Upstream had abandoned it, build tools slowly became
> incompatible with it and maintaining it became a big PITA for the KDE
> team.
>
> A lot of work had been put into making KDE3 and KDE4 co-exist already,
> since we too felt that KDE4 was not ready and I'm pretty sure we kept it
> around longer than most distros out there.
>
> But at some point KDE4 became usable and we had to move on, for our own
> sanity.

Thanks.

FWIW, I agree. It was really upstream that dropped the ball on that one,
especially after ASeigo's (in)famous promise. Both distros and users
were left to pick up the pieces on their own.

As for the timing, by the time kde3 actually headed to sunset, I had been
switched over for a couple months (maybe a bit more?) already, as I had
seen the gentoo/kde discussion and knew they were dropping it... with
little real choice given upstream kde had already dropped it, and by 4.2,
was claiming kde4 was ready for ordinary users despite it being horribly
broken alpha quality at best.

And it's a good thing I /did/ get a relatively early start, since even
tho I had been trying kde4 on and off since before its initial release,
it was simply still to broken to switch to directly, without a *LOT* of
tweaking and substitution of alternative options where there simply
wasn't a kde4 option yet (and in some cases still isn't, proper khotkeys
multi-key support being a case in point, but I ended up hacking together
a solution that worked for me). I actually switched with 4.2.5, but it
took me well over a hundred hours (that's when I stopped counting, and I
was being conservative) of stress and sweat to complete the transition.
Obviously, few would tolerate that, so it was still broken (alpha) by
definition.

Late in 4.5, say 4.5.4 or so, was the first version I considered truly
first-release quality, and that would have been a *LONG* time for a distro
to try to support kde3 without upstream itself helping. I guess Debian
stable effectively did that, but Debian stable isn't a rolling distro and
wasn't constantly upgrading the rest of the distribution out from
underneath the stale kde3, breaking it in the process, either. (Of
course, just when everything else was settling down, the kdepim devs had
to repeat pretty much the same story with akonadified kmail, in kdepim-4.6
+... the MIDDLE of what SHOULD have been a a stable-api series! Anyone
sane would have reserved that for the 4.x -> 5.x transition, but that's a
whole different story. OTOH, it was that which finally triggered my
switch off kmail/akonadi/kdepim entirely, and with that gone I no longer
had a reason to keep semantic-desktop on, so that's when I killed it here
too, thus setting the stage for this entire thread...)

On the bright side, tho, at least here all that work you put into making
kde3 and kde4 coexist reasonably peacefully was definitely NOT a waste.
It was that work, after all, that finally allowed me a successful
transition to kde4, a single app at a time, by running the kde4 version
on what began as a kde3 desktop, configuring it to usability, then
unmerging the kde3 version so the kde4 version ran by default. (Long
hairy app-by-app-transition account that was in my first post revision
elided as unnecessary...)

By switching to and configuring a kde4 app at a time, I was able to work
around two separate triggers of what turned out later to be a single root
blocker bug (the qt4-related painter bug fixed around 4.3.2 or 4.3.4),
the worst in plasma, the bad enough one in kwin, that were previously
making kde4 performance so horribly glacial that I simply couldn't work
in it long enough to configure it to my satisfaction and make the
transition. Plus some other less major problems that helped me work thru,
of course...

And if kde3 and kde4 hadn't been usable at the same time, app-at-a-time
kde4 configuration while still working in a kde3 base desktop wouldn't
have been possible. So I'm glad gentoo/kde /had/ put all the work they
did into letting them run side-by-side. =:^)

> That said, I do agree that semantic-desktop should stay optional in
> KDE4. Back in my KDE days I hated it and always ensured it - and its
> ugly dependencies - stayed off my system. Unfortunately I am not part of
> the KDE team anymore, so I don't know what their reasoning is behind
> this decision. Hopefully they will reconsider :)

Thanks.

With some luck... but I'm not counting on it. And thankfully, I'm an
experienced and well versed enough gentooer now that I /can/ handle the
patching, at least relatively short term. It's longer term that I'm
worried about.

But again with some luck, frameworks will get to be usable quickly enough
that I shouldn't have to deal with the patches for
/too/ long. All upstream indications so far is that they don't want a
repeat of the early kde4 fiasco either, and the transition should be much
smoother. That's in addition to the modularity, with the stated goal of
people being able to mix and match kde4 and kde5/frameworks apps as
desired, and only upgrade to the kde5/frameworks versions when the
individual apps are ready for it.

And there's already a very rough early alpha frameworks preview out
(AFAIK with 4.11-beta1), and a kde/kwin/wayland preview (which altho it's
a WIP, gentoo/kde /is/ apparently supporting, I see the wayland USE flags
already but haven't been brave enough to try wayland at all, yet) as well.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

kensington at gentoo

Jul 6, 2013, 6:12 AM

Post #6 of 25 (238 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

On 5/07/2013 08:02, Alex Alexander wrote:
> That said, I do agree that semantic-desktop should stay optional in
> KDE4. Back in my KDE days I hated it and always ensured it - and its
> ugly dependencies - stayed off my system. Unfortunately I am not part of
> the KDE team anymore, so I don't know what their reasoning is behind
> this decision. Hopefully they will reconsider :)
>
The reason is that code like this is appearing increasingly throughout
the KDE SC:
find_package(Akonadi REQUIRED)
set_package_properties(NepomukCore PROPERTIES TYPE REQUIRED)

That is, we must maintain hacks in order to disable upstream's
requirements. They are difficult to maintain and also put us in a bad
standing with upstream.

Best regards,
Michael

1i5t5.duncan at cox

Jul 6, 2013, 9:51 AM

Post #7 of 25 (238 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Michael Palimaka posted on Sat, 06 Jul 2013 23:12:42 +1000 as excerpted:

> On 5/07/2013 08:02, Alex Alexander wrote:
>> That said, I do agree that semantic-desktop should stay optional in
>> KDE4. Back in my KDE days I hated it and always ensured it - and its
>> ugly dependencies - stayed off my system. Unfortunately I am not part
>> of the KDE team anymore, so I don't know what their reasoning is behind
>> this decision. Hopefully they will reconsider :)
>>
> The reason is that code like this is appearing increasingly throughout
> the KDE SC:
> find_package(Akonadi REQUIRED)
> set_package_properties(NepomukCore PROPERTIES TYPE REQUIRED)
>
> That is, we must maintain hacks in order to disable upstream's
> requirements. They are difficult to maintain and also put us in a bad
> standing with upstream.

That's actually why I ended up package.providing kdebase-runtime-meta
(along with ksplash, kdesu and kdnssd, for similar not-really-necessary,
more-useless-hassle-to-upgrade when I'm running live-branch kde, reasons)
here, since it's otherwise a dependency, and it in turn pulls in drkonqi,
which at one point anyway required something kdepim related to send the
mail, and when I killed kdepim on my system, I wanted nothing to do with
that, so I package.provided kdebase-runtime-meta in ordered to be able to
use (a copy of) the kdebase-runtime set instead, with bits like drkonqi
commented so as not to bring them in. (With stripping it's not like
drkonqi ever thought the traces were usable anyway, and it's a runtime-
dep-only, that's only activated when an app crashes in any case, only to
tell me it can't use the generated reports anyway, so why even have it
installed in the first place, especially when it's pulling in some weird
kdepim dep that I otherwise can keep off my system.)

Of course that drkonqi kdepim dependency was added in one of the betas a
few feature releases ago, and as I hacked that and various other bits,
gradually adding one package.provided or patch on another to keep kde
updating, I gradually gained the experience and familiarity with kde
internals that allowed me to even /think/ about doing the whole no-
semantic patch series I'm doing now. Had things started out with that,
I'd have not thought I could, but I'm into it deep enough now, it's just
one more layer piled on the others.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

johu at gentoo

Jul 6, 2013, 10:02 AM

Post #8 of 25 (239 views)
Permalink

Re: Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Am Samstag, 6. Juli 2013, 16:51:54 schrieb Duncan:
> Michael Palimaka posted on Sat, 06 Jul 2013 23:12:42 +1000 as excerpted:
> > On 5/07/2013 08:02, Alex Alexander wrote:
> >> That said, I do agree that semantic-desktop should stay optional in
> >> KDE4. Back in my KDE days I hated it and always ensured it - and its
> >> ugly dependencies - stayed off my system. Unfortunately I am not part
> >> of the KDE team anymore, so I don't know what their reasoning is behind
> >> this decision. Hopefully they will reconsider :)
> >
> > The reason is that code like this is appearing increasingly throughout
> > the KDE SC:
> > find_package(Akonadi REQUIRED)
> > set_package_properties(NepomukCore PROPERTIES TYPE REQUIRED)
> >
> > That is, we must maintain hacks in order to disable upstream's
> > requirements. They are difficult to maintain and also put us in a bad
> > standing with upstream.
>
> That's actually why I ended up package.providing kdebase-runtime-meta
> (along with ksplash, kdesu and kdnssd, for similar not-really-necessary,
> more-useless-hassle-to-upgrade when I'm running live-branch kde, reasons)
> here, since it's otherwise a dependency, and it in turn pulls in drkonqi,
> which at one point anyway required something kdepim related to send the
> mail, and when I killed kdepim on my system, I wanted nothing to do with
> that, so I package.provided kdebase-runtime-meta in ordered to be able to
> use (a copy of) the kdebase-runtime set instead, with bits like drkonqi
> commented so as not to bring them in. (With stripping it's not like
> drkonqi ever thought the traces were usable anyway, and it's a runtime-
> dep-only, that's only activated when an app crashes in any case, only to
> tell me it can't use the generated reports anyway, so why even have it
> installed in the first place, especially when it's pulling in some weird
> kdepim dep that I otherwise can keep off my system.)
>
> Of course that drkonqi kdepim dependency was added in one of the betas a
> few feature releases ago, and as I hacked that and various other bits,
> gradually adding one package.provided or patch on another to keep kde
> updating, I gradually gained the experience and familiarity with kde
> internals that allowed me to even /think/ about doing the whole no-
> semantic patch series I'm doing now. Had things started out with that,
> I'd have not thought I could, but I'm into it deep enough now, it's just
> one more layer piled on the others.

For the record you offer nothing for users who want to use kdepim and thats
where the big story ends for us.

Greetings

--
Johannes Huber (johu)
Gentoo Linux Developer / KDE Team
GPG Key ID F3CFD2BD

Attachments:

signature.asc (0.48 KB)

1i5t5.duncan at cox

Jul 7, 2013, 1:48 AM

Post #9 of 25 (234 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Johannes Huber posted on Sat, 06 Jul 2013 19:02:55 +0200 as excerpted:

> For the record you offer nothing for users who want to use kdepim and
> thats where the big story ends for us.

Well, obviously if they want to use kdepim, they need to have the
dependencies (now) needed for it... which basically means semantic-
desktop. That's a given. But for those who steer well clear of kdepim
in part because of its heavy deps... it'd be nice to still have the
choice to /avoid/ those deps... without having to hack and patch ebuilds
to do it.

Just the usual options gentooers are used to having, both in general and
specifically with kde4, is all.

But I didn't go into this expecting to change gentoo/kde policy. I
thought I'd ask, just in case, but I didn't expect it to change. Given
that the only user response so far is (effectively) that I'm making a
mountain out of a molehill... Tho I'm not sure how many people in the
potential audience actually read this list. If I were to start a thread
on the forums and if I had added a comment to the blog-post announcing
the gentoo/kde policy change asking if there's interest there, since
there /were/ a couple comments expressing disappointment... But I'll
probably just continue doing the patches myself, until 5/frameworks gets
in good enough shape to migrate to it (assuming that's not too distant),
at which point, hopefully, upstream will be modular enough that there
won't be an issue any longer. Else, if that turns out to be too far out
and I can't maintain the patches, I'll simply find another desktop. As I
said, unlike with the early kde4 thing, that's actually a practical
option, now.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

dominique.michel at vtxnet

Jul 7, 2013, 3:10 AM

Post #10 of 25 (232 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Le Wed, 3 Jul 2013 17:32:25 +0000 (UTC),
Duncan <1i5t5.duncan [at] cox> a écrit :

> For kde-4.11, it seems the gentoo/kde project has decided to
> hard-enable the former semantic-desktop USE flag, forcing the option
> on and forcing a number of formerly optional additional
> dependencies.[1]

With USE="-consolekit -policykit -semantic-desktop -udisks
-udisks2 -upowe", I get ride of both *kit and semantic-desktop in kde,
and of the whole of gnome as a bonus. -:)

I have only parts of kde in my system, but I made such a profile for
the pro-audio overlay, and no user complained it was not working. It is
another one as generic dekstop profile.

The main concern was *kit, which is mandatory with only very few
desktops like Gnome, but is enabled into all the gentoo desktop
profiles -:(. When I see it was possible to speed up kde by removing the
semantic desktop, I did a desktop-kde profile too.

>
> But, I spent quite some time here switching away from kdepim's kmail,
> akregator, etc, so I could kill akonadi on my system, and with it
> semantic-desktop, etc, and I'm in no mood to have it hard-enabled now.
> If it comes to it, I'd rather dump the kde desktop and switch to
> something else[2], than have semantic-desktop on my system once again.
>
> But with a bit of luck, I won't have to switch away from kde after
> all.
>
> I already asked gentoo/kde to reconsider, given that they've supported
> USE=-semantic-desktop until now and with 4.11 much of kde4's going
> into maintenance mode as the upstream developer focus switches to
> kde5/kde- frameworks, so it makes little sense to drop support for
> -semantic- desktop now, when upstream is continuing to offer that
> option at least thru kde4, and kde5/frameworks is supposed to be far
> more modular, so with luck will allow users to pick and choose
> whether they want the semantic-desktop components pulled in or not.
> However, given the gentoo/ kde project history with dropping kde3
> support and forcing kde3 users to to the user-supported kde-sunset
> overlay even while kde4 was still not ready for use (and despite
> upstream kde's broken promise to support kde3 as long as there
> continued to be users), I'm not optimistic, but it was worth a shot.
>
> But the kde-sunset overlay does suggest another alternative, a kde4-
> nosemantic overlay.
>
> Meanwhile, as I upgraded to the kde-4.11 pre-releases (currently
> 4.10.90 aka 4.11-beta2) in the kde overlay, for the kde-desktop-core
> and other gentoo/kde packages I still run, I diffed the ebuilds
> between 4.10.x and 4.10.80 (aka 4.11-beta1), then checked the diffs
> for non-semantic-desktop related changes and kept them, while
> changing the semantic-desktop force- enabling changes to
> force-disabling instead.
>
> Then I created a framework that works much like epatch_user, except
> instead of automatically applying patches to upstream sources, it
> automatically applies patches to gentoo ebuilds and instead of using
> the /etc/portage/patches/ tree, it uses /etc/portage/patches.ebuild/.
>
> So now I have a set of ebuild patches that patch the kde 4.11 ebuilds
> (starting with 4.10.80, aka 4.11-beta1) to force-disable semantic-
> desktop, instead of force-enabling it. And I have a scripted
> framework that auto-applies these patches to new ebuilds on emerge
> --sync and layman -S, thus keeping no-semantic around as upstream
> gentoo/kde updates their ebuilds.
>
> For now, therefore, I'm fine, up and running on 4.10.90 (aka
> 4.11-beta2), using gentoo/kde ebuilds auto-patched to kill the now
> forced-on semantic- desktop, forcing it off instead.
>
> But realistically, I honestly don't know if longer term, I'll be able
> to continue maintenance of all of this by myself. Chances are
> unfortunately high that without help from others, over time I'll
> decide it's simply too much of a hassle maintaining the patches, and
> will end up switching to some other desktop, with the qt-based
> razor-qt desktop one candidate as sort of a kde-lite desktop, and
> enlightenment as another, getting away from kde and qt entirely.
>
> Besides which, if I'm finding kde-nosemantic useful enough to go to
> all this trouble, there's a good chance that others will be
> interested in it themselves, especially if they don't have to do all
> the work I'm now doing myself, themselves. So with kde-sunset in
> mind as precedent, I'm now proposing a kde-nosemantic overlay, like
> kde-sunset, user-maintained, but for kde4 folks who want a continued
> no-semantic choice, instead of kde3 users.
>
> Any interest?
>
> To be further discussed: Assuming a go-ahead on the general idea, do
> we want to maintain it as a normal overlay carrying at least the kde4
> ebuilds that require patching to kill semantic-desktop, or should we
> simply build on the epatch_ebuild_user scripts I have hacked up,
> presumably checking them into a git repo along with the patches
> themselves somewhere and making that available, then simply use that
> tool with the existing gentoo tree (when 4.11 is released and ebuilds
> arrive in the main tree) and kde project overlay to apply the patches
> to the existing tree and overlay instead of creating a full-fledged
> kde-nosemantic overlay ourselves. Of course the tools and patches
> could then have ebuilds and appear in an overlay of their own, rather
> than having the modified kde-nosemantic ebuilds in an overlay.
>
> One bonus to the tools overlay instead of a direct kde-nosemantic
> overlay approach, is that gentooers not interested in kde, but
> interested in the ebuild-patch tools, might find that useful, add
> that overlay to their layman overlay list, and contribute patches to
> the ebuild-patches tool, helping it mature and grow into a general
> purpose automated-ebuild- patching tool rather faster than it might
> otherwise happen.
>
> A hybrid alternative would be to adopt an idea much like the existing
> kde overlay, where there's a documentation or tools directory that
> carries them, in addition to the kde-base category and etc, carrying
> the patched ebuilds themselves.
>
> So what do people think? Any interest? How should we go about it?
>
> Or should I just continue working on it on my own, with the likelihood
> that at some point I'll decide it's not worth the trouble and switch
> to a non-kde desktop, as I've switched to other non-kde tools as the
> kde versions jumped the shark over the course of kde4?
>
> In particular, I expect users who are or have been active in the kde-
> sunset overlay will have some useful insights.
>
> ---
> [1] Andreas Huttel, aka gentoo dev dilfridge, covered this on his blog
> (which is in turn covered by planet-gentoo, where I subscribe to the
> feed, thus seeing it there):
>
> http://dilfridge.blogspot.com/2013/05/news-from-201305-gentoo-kde-team-meeting.html
>
> [2] While during the early kde4 fiasco I was mostly standardized on
> kde apps and therefore had little choice, over the course of kde4,
> I've switched away from kde apps for first one thing than another, so
> by now it's mostly the core kde4 desktop I depend on, plus a few
> other less vital apps, games, dolphin, gwenview, superkaramba, that I
> could leave behind far more easily now, if I decided I could no
> longer run the kde- core-desktop.
>

--
"We have the heroes we deserve."

1i5t5.duncan at cox

Jul 7, 2013, 2:41 PM

Post #11 of 25 (220 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Dominique Michel posted on Sun, 07 Jul 2013 12:10:29 +0200 as excerpted:

> With USE="-consolekit -policykit -semantic-desktop -udisks -udisks2
> -upowe", I get ride of both *kit and semantic-desktop in kde, and of the
> whole of gnome as a bonus. -:)

That's very similar (identical?) to what I have. But the semantic-
desktop flag is going away for kde 4.11... unfortunately, and they're
forcing semantic-desktop on. The gentoo/kde project reasoning is in the
link I provided in the thread starter.

You might try razor-qt or a gtk-based desktop instead. Or... my patches.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

dominique.michel at vtxnet

Jul 8, 2013, 9:41 AM

Post #12 of 25 (216 views)
Permalink

Re: Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Le Sun, 7 Jul 2013 21:41:36 +0000 (UTC),
Duncan <1i5t5.duncan [at] cox> a écrit :

> Dominique Michel posted on Sun, 07 Jul 2013 12:10:29 +0200 as
> excerpted:
>
> > With USE="-consolekit -policykit -semantic-desktop -udisks -udisks2
> > -upowe", I get ride of both *kit and semantic-desktop in kde, and
> > of the whole of gnome as a bonus. -:)
>
> That's very similar (identical?) to what I have. But the semantic-
> desktop flag is going away for kde 4.11... unfortunately, and they're
> forcing semantic-desktop on. The gentoo/kde project reasoning is in
> the link I provided in the thread starter.
>
> You might try razor-qt or a gtk-based desktop instead. Or... my

Thanks, I am happy with fvwm-crystal from years. And I am not
interested to maintain patches in the pro-audio overlay for a desktop
I will not use anyway.

Dominique

> patches.
>

--
"We have the heroes we deserve."

vaeth at mathematik

Jul 11, 2013, 12:25 AM

Post #13 of 25 (208 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Duncan <1i5t5.duncan [at] cox> wrote:
>
> Given
> that the only user response so far is (effectively) that I'm making a
> mountain out of a molehill...

I just post to let you know that you are not alone :)

You also have friends in the forum sharing your opinion
https://forums.gentoo.org/viewtopic-t-963504-highlight-.html

Your effort is really appreciated.
However, I guess that most people are like me and just gave up:
If it really means to make new upstream patches and actually
fight against upstream policy, it is not worth the hassle.
The change of the Gentoo policy caused me to remove KDE,
and I guess most users who do not want the dependencies
have done (or will do) the same.

Independently on the overlay, I think your framework is
very interesting: Does your framework manage the ebuilds
in some overlay, or is it actually running tranparently
during emerge (probably with a patched version of portage),
allowing e.g. to change metadata without maintaining the
ebuild in a separate local overlay?
(I guess that it is the former, but your choice of the path
/etc/portage/... suggests the latter)

1i5t5.duncan at cox

Jul 11, 2013, 6:25 AM

Post #14 of 25 (207 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Martin Vaeth posted on Thu, 11 Jul 2013 07:25:37 +0000 as excerpted:

> Independently on the overlay, I think your framework is very
> interesting:
> Does your framework manage the ebuilds in some overlay, or is it
> actually running tranparently during emerge (probably with a patched
> version of portage), allowing e.g. to change metadata without
> maintaining the ebuild in a separate local overlay?
> (I guess that it is the former, but your choice of the path
> /etc/portage/... suggests the latter)

To be explicit, I'm using the official gentoo/kde overlay, which of
course is the only place with kde 4.11 series (4.10.80+) ebuilds ATM,
since 4.11 is still pre-release and we're dealing with the beta (4.10.80,
4.10.90) and rc (4.10.95+) sources and ebuilds. But the framework I've
setup is repo agnostic and would find the ebuilds in whatever repo, main
tree or active overlay, they appear in.

Some background...

For some years now some ebuilds have made use of the epatch_user function
from eutils.eclass or base.eclass. That function made use of a patches
tree organized by category and package at /etc/portage/patches/,
automatically applying any patches it found in the directory
corresponding to the ebuild being run. Originally, calling epatch_user
was optional -- the ebuild had to inherit the appropriate eclass and call
the function. However, with eapi-5, it's now a mandatory part of any
eapi-5 compliant ebuild.

Actually, I believe that idea originated with a guy (I've forgotten his
name and AFAIK he's no longer a gentooer, IIRC his domain expired some
time ago and I deleted the bookmark I had to it) who had a patched
portage with hooks at selected spots in the code, and various optional
utilities using those hooks for all sorts of useful stuff. It seems
enough gentoo devs found his stuff useful, that portage gradually
integrated many of his hooks and tools, including this one.

Meanwhile, long before eapi-5 came along, I got tired of having to figure
out whether dropping a patch in the /etc/portage/patches tree was going
to work for a particular ebuild or not, and hacked up something using
/etc/portage/bashrc and a couple stub scripts, that ensured that I had
epatch_user available as a function, and ran it using one of the hooks
portage integrated for such extensions, the post_src_prepare function, as
I defined it in /etc/portage/bashrc. So for some time now, I've been
able to simply drop source patches in the appropriate /etc/portage/
patches/ subdir and have them automatically applied, regardless of
whether the ebuild actually called epatch_user or not.

As anyone who for example tests gcc versions before they're unmasked will
know, a patch tree of this nature comes in really handy for applying
various source patches without having to manually modify the ebuild. =:^)

But, while that works great for source patches, at times one has to
modify ebuilds too, and there isn't yet an offical similar framework for
automatically applying ebuild patches.

I've occasionally been frustrated by this, but until this gentoo/kde
semantic-desktop policy change, particularly with epatch_user eliminating
the need for most of the ebuild edits I used to do, it was just easier to
do the manual hacking any time I needed to change an actual ebuild.

But with the gentoo/kde semantic-desktop policy change, that too changed,
as I was now doing ebuild patching longer term and on a rather wider
scale. So I needed something like epatch_user, but for the ebuilds
themselves instead of for the sources the ebuilds used.

Meanwhile, some time ago I setup a script that among other things ran
emerge sync and layman -S (in parallel), then ran emerge --regen (with
multiple jobs) to rebuild the cache to take into account layman's
overlays. Over time, this script expanded a bit to take on additional
tasks, including checking to see if my portage and sources partition was
mounted before running the sync and mounting it if not, updating the
esearch and portage-utils databases, doing an emerge --update --deep
--fetch @world, etc. The script is called esyn (esync without the
terminating c), and I run it instead of esync to automatically take care
of all the stuff I'd otherwise do one command at a time at the beginning
of an update run.

So when the need for an ebuild-patching variant of the epatch_user
function came up with this gentoo/kde policy change and my subsequent
generation of all these ebuild patches I needed applied, it was rather
natural to simply add another function to my esyn script, that after
the syncs looped thru a tree paralleling /etc/portage/patches (which I
chose to call /etc/portage/patches.ebuild), and upon finding a patch in
that tree, looped thru each of the repos listed in $PORTDIR and
$PORTDIR_OVERLAY (as set in make.conf), matching category and package to
see if there are any matching ebuilds in that repo to patch.

If it finds a matching ebuild, like epatch_user, it first tries applying
the patch in a dry-run. If the patch applies in the dry-run, then it
applies it for real. If the patch doesn't apply in the dry-run, then it
could be an ebuild for an old version (in this case, kde 4.10 ebuilds
still have the semantic-desktop USE flag and don't need patched, neither
will the patches apply), or it might be an ebuild that had the patch
applied already (unlike with rsync, unless there's a conflict, git
doesn't overwrite local changes, so the patch stays applied at layman -S
and an attempted re-apply will fail; if there's a conflict, I can git
reset --hard the overlay and redo the overlay sync to pull down the
update, after which, given the conflict with the patch I had previously
applied, I'll likely need to create an updated patch to apply), etc, so
the function simply skips that ebuild and moves on to the next.

At this point it's all rather hacked together, but it works here. I
expect that over time, as the ebuilds from the gentoo/kde overlay and
eventually in the main tree change, I'll find the patches don't apply and
my updates break, at which point I'll update the patches and/or update
the ebuild patching function as necessary. Based on past experience, on
my own I'd get something semi-robust for my own setup in a few months to
a year, tho it still wouldn't necessarily work well for others. Of
course if I were to turn the whole thing into a publicly available
project and take patches or even make it a publicly managed project (much
like kde-sunset), I expect it'd mature much faster and would be become
rather less hacky and rather more general purpose relatively quickly,
such that with the help of others it'd likely be generally usable and
reasonably stable within a couple months, instead of the year or so it'll
likely take me on my own, after which it'll be rather more robust but
still be targeted at only my system, if I don't take it public.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

slong at rathaus

Jul 15, 2013, 7:01 PM

Post #15 of 25 (201 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Duncan wrote:
> Martin Vaeth posted:
> > Does your framework manage the ebuilds in some overlay, or is it
> > actually running tranparently during emerge (probably with a patched
> > version of portage), allowing e.g. to change metadata without
> > maintaining the ebuild in a separate local overlay?
> > (I guess that it is the former, but your choice of the path
> > /etc/portage/... suggests the latter)

> the framework I've setup is repo agnostic and would find the ebuilds in
> whatever repo, main tree or active overlay, they appear in.

> I've occasionally been frustrated by this, but until this gentoo/kde
> semantic-desktop policy change, particularly with epatch_user eliminating
> the need for most of the ebuild edits I used to do, it was just easier to
> do the manual hacking any time I needed to change an actual ebuild.

Hmm why was a local overlay inappropriate? I thought you could mask packages
from specific overlays (or am i wrong about that?)

Not that it matters, in that I'm glad you've gone down this path. I'd just
like to know.

From what you've written, the first thing that springs to mind is
/etc/portage/postsync.d/ which has q-reinitialize in it. I have that activated,
and run eix-sync after emerge --sync in update, which takes care of overlays.
Which answers why postsync isn't sufficient in and of itself. Hmm I guess that
means that means qgrep et al aren't complete, which I've never noticed as I don't
use overlays.

Additionally, like Martin, I assumed you were doing this in a local overlay, not
on the tree itself, with resultant rsync wipeout. So if we can sort that out, by
writing the output to:
"${local_overlay:=/usr/local/portage}/${ebuild#"$PORTDIR"}"
I'd be a lot happier. It'll fit much better in the process of pushing to an
external overlay as well.

The other thing I was going to mention is that update -s wraps the whole process,
to filter the rsync output so it's only one line (the performance thing I mentioned
in the other post: I never expected that to work so well but my mentor does the awk
so I just tried in bash, and was amazed. :) So we could:
a) get the list of ebuilds that have actually been changed via rsync, or
b) check what eix is reporting after (if the user has eix.)

However a postsync action similar to q would be the best; it just has to run after
the overlays have been sync'ed. As I said I don't use overlays, so it's up to you
and people like Martin to work the details of 'what and when' out. I'll help with
the 'how'. I'd just prefer something that doesn't require a wrapper, but works with
any emerge setup.

Alternatively, if we can get eix to do it directly, it'd rock afaic. Though a
developer I know refuses to use it, as he says it takes too long if you've got
a cvs tree, or sth. Unfortunately, like you, mv doesn't do IRC, so I've not been
able to get the two together for a chat to see if anything could be sorted, or
whether it's intractable.

Personally, I'm fine with a dep on eix fwiw.

I'm reasonably sure you can hook whatever you like into eix-sync, as I recall
telling people to just use it quite a lot back in the day (we do have postSync
actions as well, one for -q quiet usage, if defined) but mv can tell us more, if
eix can help. It seems the right spot, since eix-sync can run after emerge --sync
and do all the overlays, so people are used to running it. iirc again you can use
a postsync.d action, so it would be ideal.

We just don't with update as we want the rsync filtering, and have to work when
the user doesn't have eix, so it's always been separate. Having said that, if the
user has eix running in postsync, that'll work too.

That's the trouble with glue-scripting: you have to consider the interaction of
quite a few disparate commands, and various end-user setups.

It's also what makes it useful, and fun to work on. :-)

--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

vaeth at mathematik

Jul 16, 2013, 9:35 AM

Post #16 of 25 (200 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Steven J. Long <slong [at] rathaus> wrote:
>
> From what you've written, the first thing that springs to mind is
> /etc/portage/postsync.d/ which has q-reinitialize in it. I have that
> activated, and run eix-sync after emerge --sync in update, which takes
> care of overlays.
> Which answers why postsync isn't sufficient in and of itself.

I don't understand why postsync.d is not sufficient and why you run
eix-sync *after* emerge --sync (it should be run *instead of*).
eix-sync alone normally uses this order (only relevant tasks listed):

1. layman ...
2. emerge --sync [ hence, followed by postsync.d hooks ]
3. @-hooks from /etc/eix-sync.conf
4. eix-update
5. eix-diff

so if you use postsync.d to update a local overlay according to changes in
the tree (or of a layman overlay) this update should be visible in eix.
If you do not use eix, postsync.d should do as well...

If you want to avoid postsync.d (though I still do not understand why)
and use eix directly you can use the @-hooks:
Put e.g. into /etc/eix-sync.conf the lines

@StatusInfo Updating local overlay
@/path/to/command_to_update_local_overlay

1i5t5.duncan at cox

Jul 17, 2013, 4:28 AM

Post #17 of 25 (199 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Steven J. Long posted on Tue, 16 Jul 2013 03:01:50 +0100 as excerpted:

> Duncan wrote:
>> Martin Vaeth posted:
>> > Does your framework manage the ebuilds in some overlay, or is it
>> > actually running tranparently during emerge (probably with a patched
>> > version of portage), allowing e.g. to change metadata without
>> > maintaining the ebuild in a separate local overlay?
>> > (I guess that it is the former, but your choice of the path
>> > /etc/portage/... suggests the latter)
>
>> the framework I've setup is repo agnostic and would find the ebuilds in
>> whatever repo, main tree or active overlay, they appear in.
>
>> I've occasionally been frustrated by this, but until this gentoo/kde
>> semantic-desktop policy change, particularly with epatch_user
>> eliminating the need for most of the ebuild edits I used to do, it was
>> just easier to do the manual hacking any time I needed to change an
>> actual ebuild.
>
> Hmm why was a local overlay inappropriate? I thought you could mask
> packages from specific overlays (or am i wrong about that?)

I believe I may have misunderstood Martin's original comment intent. The
below might more accurately address the situation. (Assuming I'm not
getting too sleepy to think straight...)

I do have a local overlay, and use it for one-offs, where the upstream
ebuild's likely to include the patch relatively quickly, or where it only
applies to a specific version so an update is likely to invalidate the
patch in any case.

But in the kde case that wouldn't work as the patches will need to be
reapplied long-term -- no upstream inclusion, as I didn't want to deal
with manually overlaying/patching every single revision bump, etc.

Actually, for kde I run the betas from the overlay, and (since sometime
in 4.10) have been running (and generally rebuilding about twice a week)
the live-branch ebuilds (4.10.49.9999, 4.11.49.9999) when upstream
branches along with the first rc. Particularly the live-branch ebuilds
(and even more the live-trunk -9999 ebuilds, but I've not gotten /that/
brave yet) get updated in-place to adapt to upstream code changes as well
as gentoo system changes, and I wanted my hassle-free application of my
no-semantic patches until they no longer applied, at which point I wanted
to know about it so I could redo them.

So applying the patches direct-in-repo made most sense for me, with
updates overwriting my changes, and the patches then reapplied on top of
the updates.

But making that an option's probably a good idea, agreed.

> Not that it matters, in that I'm glad you've gone down this path. I'd
> just like to know.
>
> From what you've written, the first thing that springs to mind is
> /etc/portage/postsync.d/ which has q-reinitialize in it. I have that
> activated, and run eix-sync after emerge --sync in update, which takes
> care of overlays. Which answers why postsync isn't sufficient in and of
> itself. Hmm I guess that means that means qgrep et al aren't complete,
> which I've never noticed as I don't use overlays.

There's also the fact that my patches change dependencies -- that's what
they're DESIGNED to do, after all, kill the semantic-desktop deps -- thus
invalidating portage's metadata cache, so if emerge --regen isn't
triggered afterward, every emerge invocation will take longer.

What my sync script (local version) basically does:

1) test-me: see if the tree partition is mounted and mount it if
necessary.

2) setup-parms: grab niceness and parallel jobs from my portage
configuration, etc.

3) emerge sync and layman sync (backgrounded in parallel, using wait).

4) ebuild-patch (that's my ebuild patching function, the interesting bit).

5) emerge $EJobs --regen

Then after the portage cache is updated, in background/parallel:

6) emerge --fetch (options) @world

7) eupdatedb (for esearch, with portage niceness applied)

8) q -qr (with portage niceness applied)

9) Then (while 6-8 are still backgrounded), spit out a "syncs done,
fetches and db updates continuing in background" message, after which it
returns me to a prompt while the background jobs complete.

Obviously that'll need generalized for non-local use. 1 and 2 could be
generalized into presync.d (ordered), 6-9 into postsync.d (parallel), 3
into simply sync.d (parallel), and 4 and 5 into immediate-postsync.d (or
some such, ordered).

Then people could simply drop scriptlets into the appropriate *sync.d dir
and let the general script handle it. Meanwhile, #4, the ebuild-patch
step that's of interest here, would become just another scriptlet file in
immediate-postsync.d.

> Additionally, like Martin, I assumed you were doing this in a local
> overlay, not on the tree itself, with resultant rsync wipeout. So if we
> can sort that out, by writing the output to:
> "${local_overlay:=/usr/local/portage}/${ebuild#"$PORTDIR"}"
> I'd be a lot happier. It'll fit much better in the process of pushing to
> an external overlay as well.

An option makes sense...

> The other thing I was going to mention is that update -s wraps the whole
> process, to filter the rsync output so it's only one line (the
> performance thing I mentioned in the other post: I never expected that
> to work so well but my mentor does the awk so I just tried in bash, and
> was amazed. :) So we could:
> a) get the list of ebuilds that have actually been changed via rsync, or
> b) check what eix is reporting after (if the user has eix.)

I think the generalized *sync.d approach should still be wrappable. And
anything the wrapper handles can simply be left out of the *sync.d dir
it'd otherwise be located in.

> That's the trouble with glue-scripting: you have to consider the
> interaction of quite a few disparate commands, and various end-user
> setups.
>
> It's also what makes it useful, and fun to work on. :-)

Indeed. =:^)

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

vaeth at mathematik

Jul 18, 2013, 7:32 AM

Post #18 of 25 (195 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Duncan <1i5t5.duncan [at] cox> wrote:
>
> I do have a local overlay, and use it for one-offs

You can have several local overlays, one particularly dedicated for KDE
(and if you put the directory of the latter under git control,
you could also easily make it public e.g. on GitHub so that other users
can use it without using your framework).

> But in the kde case that wouldn't work as the patches will need to be
> reapplied long-term -- no upstream inclusion, as I didn't want to deal
> with manually overlaying/patching every single revision bump, etc.
> [...] So applying the patches direct-in-repo made most sense for me

Why not use the script to patch the ebuilds after fetching
but store the patched ebuilds in your dedicated overlay instead
of the original location?
If you give this dedicated overlay a higher priority in
/etc/portage/repos.conf, portage will install the patched
ebuilds if they are available.

For a general framework, one could e.g. support directories
of the form

/etc/portage/ebuild.patches/FROM:TO/whatever

so that "whatever" (patches, sed-commands, etc; depends how
your framework works) is applied by your framework after
it copied the corresponding ebuild from repository FROM to TO.
In particular, currently you would use only something like

/etc/portage/ebuild.patches/kde-experimental:kde_unsemantic/...

The scripts in your framework can use functions like
portageq get_repo_path / FROM
or the quicker new
eix-header -p FROM
to find out the corresponding paths, once these repositories
are set up (and in the eix database, respectively).

> There's also the fact that my patches change dependencies -- that's what
> they're DESIGNED to do, after all, kill the semantic-desktop deps -- thus
> invalidating portage's metadata cache, so if emerge --regen isn't
> triggered afterward, every emerge invocation will take longer.

For a local overlay you do not need to run emerge --regen. Running

egencache --repo=kde_unsemantic --update --update-use-local-desc

after applying the patches is sufficient: If kde_unsemantic has
a higher priority in /etc/portage/repos.conf, its metadata will
be taken.
BTW, I would suggest to put into metadata/layout.conf of
kde_unsemantic the lines

cache-formats = md5-dict
thin-manifests = true

and if you use git also into .gitignore the line

/metadata/md5-cache/

so that users have to run the above egencache command on their own
(which is better than having possibly outdated checksums for
eclasses in which case md5-cache would not help them, anyway).

>> That's the trouble with glue-scripting: you have to consider the
>> interaction of quite a few disparate commands, and various end-user
>> setups.

That's why for end-users publishing the patched overlay would
be better: They can still come up with patches anyway, i.e.
they are not even excluded from development if they do not use
your framework.

slong at rathaus

Jul 18, 2013, 12:30 PM

Post #19 of 25 (195 views)
Permalink

Re: kde-lean overlay [In reply to]

Martin Vaeth wrote:
> Steven J. Long wrote:
> >
> > From what you've written, the first thing that springs to mind is
> > /etc/portage/postsync.d/ which has q-reinitialize in it. I have that
> > activated, and run eix-sync after emerge --sync in update, which takes
> > care of overlays.
> > Which answers why postsync isn't sufficient in and of itself.
>
> I don't understand why postsync.d is not sufficient and

> why you run
> eix-sync *after* emerge --sync (it should be run *instead of*).

You don't appear to have been reading, which is odd, as the mail you got
that info from explained the background at the same time. In summary, again:
we wrap emerge rsync, so it only takes one line on the terminal, and then
disappears. At the time I wrote it, I tried hard to get eix to do everything,
believe me.

Additionally,remember that I've had some people say they won't use eix (again as
mentioned prior, though you appear to have started to address the issue in your
more recent mail.) So irrespective of how nice eix is, we have to support users
without it. And at the time (5 years ago) it was a lot simpler to keep the
separation.

After all, there's not much eix can do about network speed, is there?
So what exact benefit do I get by hard-depending on eix, and losing utility?
It's the same runtime.

Don't get me wrong. I regularly sing the praises of eix, and quite often whinge
that someone should just work with you properly, in #gentoo-portage, no doubt to
their annoyance. But I don't code python, nor am I about to start, and you don't
do IRC, so there's not much I can do about it.

But in general we filter most of emerge's messages, in case there's something
important like a news item, leftover config-updates, a profile upgrade, package
moves and so on. And any message emerge spits out as IMPORTANT. Those can come
up during sync as well.

> eix-sync alone normally uses this order (only relevant tasks listed):
>
> 1. layman ...
> 2. emerge --sync [ hence, followed by postsync.d hooks ]
> 3. @-hooks from /etc/eix-sync.conf
> 4. eix-update
> 5. eix-diff

Well we do emerge --sync, then by default run: eix-sync '-0 -N'

That's changed once before, a few years ago, actually after discussion with you,
around the metadata sync times. But I'm happy to change the default flags again,
if you recommend something else. As is, it's always been a config setting, and it's
up to the user to explore the wider Gentoo landscape. In this case that's the massive
manpages for eix.

If you want us to run it in a different order, that's of course fine too. I don't
use overlays, I just like the tree diff from eix, and the speed of querying.
Note the user can override with postSync hooks as well. If it's layman --sync for
example, it gets run as the wrapped layman --sync command, after eix-sync. The idea
there is that a user who is using eix doesn't set that, and instead tells eix to do
it all, which is what we've been advising since 2007/8. But another user might.

(or ofc it can be a user-defined function.)

We don't shield the user from emerge: that's not update's purpose. It's designed to
make the routine decisions easier, that belong in a higher layer, closer to the user
and not to the package manager: ie scripted. An example of that is, we don't support
uninstall: in a few places we just tell the user to run emerge -Cq package (like
update -h/--help, and iirc in depclean or one of those maintenance tasks.)

And to give you all the info at the time you're making a decision. But we take the
position that dumbing-down Gentoo is a disservice to everyone: all you're doing is
postponing the inevitable, and the user who's done their own manual install, and knows
how and where to setup files, is much more likely to stay the distance, and be able to
recover when the idiot-box is being an idiot.

That's why coloured diffs of the relevant files are nice, since it reminds you where
stuff goes on, at the same time as you're confirming changes. Of course, if you want
to get complex, it'd probably take as long as trying to explain eix, given the amount
of additions that have been put in over the years, for binhosts, tinderboxes, chroots
etc.

> so if you use postsync.d to update a local overlay according to changes in
> the tree (or of a layman overlay) this update should be visible in eix.
> If you do not use eix, postsync.d should do as well...
>
> If you want to avoid postsync.d (though I still do not understand why)

Again, that's because you're reacting to certain statements without considering
the whole. Likely because they're things that annoy you, with many of your users.
I know the feeling ;)

The simple fact is I raised postsync immediately, and was still musing as to how
we could use it. But as you've shown above, there's a lot of stuff that goes on
after the postsync (and overlay's have to be considered.)

That's why I've always recommend our users use eix to manage their overlays. We
wrap layman sync as well, but for end-users our advice has ALWAYS been: use eix.

I have no clue where Duncan's stuff comes in as yet, and it still sounds like it
needs reworking and QA once it's actually producing an overlay, as opposed to gobbing
all over the local mirror. If the process can run in postsync, that would be ideal.
As I already stated.

Seriously, I'm on your side. I've personally been using eix to display the tree, and
gladly, since the beginning.

So chill out ;)

--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

kensington at gentoo

Jul 25, 2013, 11:00 AM

Post #20 of 25 (174 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

I just saw that in Arch Linux's AUR[1], they provide a fake (empty)
package that satisfies the package manager's dependencies, but doesn't
actually do anything.

Perhaps this approach could help ease the maintenance burden?

[1]: https://aur.archlinux.org/packages/nepomuk-core-fake/

1i5t5.duncan at cox

Jul 25, 2013, 11:26 AM

Post #21 of 25 (175 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Michael Palimaka posted on Fri, 26 Jul 2013 04:00:30 +1000 as excerpted:

> I just saw that in Arch Linux's AUR[1], they provide a fake (empty)
> package that satisfies the package manager's dependencies, but doesn't
> actually do anything.
>
> Perhaps this approach could help ease the maintenance burden?
>
> [1]: https://aur.archlinux.org/packages/nepomuk-core-fake/

The way gentoo (or rather, portage) does that is package.provided... but
that doesn't take care of ebuilds hard-enabling build-time deps, which
then cause the build to fail when they're not found. Those hard enablings
must be patched out, and if that's being done, might as well patch out
the dependency itself at the same time.

Binary-based packages don't have the build-time problem, but they /can/
have other problems if they were built against libraries that simply
aren't there to provide their symbols to load.

A stub library could be built to provide the symbols and short-out the
logic as necessary, but that's beyond /my/ expertise, anyway.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

kensington at gentoo

Jul 26, 2013, 6:36 AM

Post #22 of 25 (171 views)
Permalink

Re: Interest inquery: kde4-nosemantic overlay [In reply to]

On 26/07/2013 04:26, Duncan wrote:
> but
> that doesn't take care of ebuilds hard-enabling build-time deps, which
> then cause the build to fail when they're not found. Those hard enablings
> must be patched out, and if that's being done, might as well patch out
> the dependency itself at the same time.
Which ebuilds hard-enable?

fabianoengler at gmail

Jul 26, 2013, 6:36 PM

Post #23 of 25 (169 views)
Permalink

Re: Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Oh man, knowing the next release will force enable semantic desktop is bad
news, I would surely drop KDE desktop when that became stable and I am
forced to update.

Your work offers some hope though, I think the better approach for end
users would be a plug and play overlay, but if I haven't read it here I
would probably not even think about looking for an overlay for this and
would just move away, which I suspect some other users will do too..

Thanks for the time and effort into sharing this, much appreciated.

Fabiano.
On Jul 11, 2013 5:05 AM, "Martin Vaeth" <vaeth [at] mathematik>
wrote:

> Duncan <1i5t5.duncan [at] cox> wrote:
> >
> > Given
> > that the only user response so far is (effectively) that I'm making a
> > mountain out of a molehill...
>
> I just post to let you know that you are not alone :)
>
> You also have friends in the forum sharing your opinion
> https://forums.gentoo.org/viewtopic-t-963504-highlight-.html
>
> Your effort is really appreciated.
> However, I guess that most people are like me and just gave up:
> If it really means to make new upstream patches and actually
> fight against upstream policy, it is not worth the hassle.
> The change of the Gentoo policy caused me to remove KDE,
> and I guess most users who do not want the dependencies
> have done (or will do) the same.
>
> Independently on the overlay, I think your framework is
> very interesting: Does your framework manage the ebuilds
> in some overlay, or is it actually running tranparently
> during emerge (probably with a patched version of portage),
> allowing e.g. to change metadata without maintaining the
> ebuild in a separate local overlay?
> (I guess that it is the former, but your choice of the path
> /etc/portage/... suggests the latter)
>
>
>

dilfridge at gentoo

Jul 27, 2013, 9:04 AM

Post #24 of 25 (166 views)
Permalink

Re: Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Am Samstag, 27. Juli 2013, 03:36:08 schrieb Fabiano Engler:
> Oh man, knowing the next release will force enable semantic desktop is bad
> news, I would surely drop KDE desktop when that became stable and I am
> forced to update.

One of the pertinent facts about the Gentoo mailing lists is that regularly
molehills are percieved and described as mountains.

[.Yes you may quote that for LWN distro quote of the week. If you must.]

--

Andreas K. Huettel
Gentoo Linux developer
dilfridge [at] gentoo
http://www.akhuettel.de/

Attachments:

signature.asc (0.94 KB)

slong at rathaus

Jul 28, 2013, 8:28 AM

Post #25 of 25 (128 views)
Permalink

Re: Re: Interest inquery: kde4-nosemantic overlay [In reply to]

Andreas K. Huettel wrote:
> Also schrieb Fabiano Engler:
> > Oh man, knowing the next release will force enable semantic desktop is bad
> > news, I would surely drop KDE desktop when that became stable and I am
> > forced to update.
>
> One of the pertinent facts about the Gentoo mailing lists is that regularly
> molehills are percieved and described as mountains.

Far more pertinent for a user, is that your concerns will regularly be disparaged
by "developers" who don't really see their mission as enabling users to get the
job done.

Consequently you may expect sneering or sardonic replies to valid issues that you
raise, even when you're not arguing with their decision, but simply providing
workarounds to enable people to use their machines how they see fit.

When pushed beyond condescension, the usual response is whining about "use-cases"
and questioning of your motives and method, since you can "just" turn it off, or
some other guff that ignores your valid reasons for wanting another option, and
consequently does not help you to get the job done.

Still, you likely have more experience than the person patronising you, and are
grown-up enough to shrug it off and get on with the task at hand.

That doesn't mean you have to like it, and no-one external will expect you to:
just the usual clique that will round on you for daring to step into the snakepit
and speak your mind.

Ignore them, and do not respond for a couple of days, and never in kind, as blame
will only ever attach to you, irrespective of how rude they might be.

> [.Yes you may quote that for LWN distro quote of the week. If you must.]

If only it were worth quoting.

--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)

Index | Next | Previous | View Threaded

Interested in having your list archived? Contact Gossamer Threads