How to add pl.* hierarchy to a news server

This document contains the most important things related to adding the pl.* news hierarchy to a news server or just configuring the server to carry it properly if pl.* groups are already there. It is intended mainly for news system administrators (and Polish users abroad trying to convince their news administrators to get "pl.*" groups). To make things easier, this is a detailed step-by-step instruction, but if it seems too elementary for you - look at the end for a quick checklist (Use your favourite key sequence to find '=====').

If you are a news administrator at some site and somebody has asked you to participate in distribution of pl.* groups, then read on.

If you are a news user, who wants to read the pl.* groups on his default news server - you should find the news administrator, kindly ask him if he could arrange receiving the pl.* newsgroups, and send him this article (or the pointer to the HTML version at http://www.usenet.pl/doc/news-pl-new-site-faq.html), to make his job easier.


There are few things to do, if you are going to receive the pl.* news hierarchy, but the most important ones are:

You may also find answers to some general questions about pl.* groups at the end of this document:


1. Getting the articles.

Contact the news administrator of the news server already carrying pl.* groups. Try to find the nearest one (try 'ping -s' and 'traceroute'). The best of course is when pl.* groups are carried over your 'normal' feed - ie. together with other news hierarchies. But as the pl.* news are quite young (first groups were created about June 1994), there are not many news servers outside of Poland having them, so you may have to create a special new feed for them. Below is the list of some servers outside Poland (and some in Poland) carrying pl.*, that I know of. It is not very long (nor complete) as you see. If the contact address is not explicitly listed, try 'usenet' at the news server's address, as 'usenet@news.ict.pwr.wroc.pl'.

USA             uunet.uu.net          (X 1995)
USA             gail.ripco.com
USA/MCI         news.mcs.com
USA             ratty.wolfe.net 
USA/Sprintlink  news.sprintlink.net
USA             news.hookup.net
USA             news.uoregon.edu      newsadmin@news.uoregon.edu
USA             news.efn.org          Russ Taylor (17.12.1996)
Canada/Toronto  news.ki.net           Marc G. Fournier <news@ki.net> (5.08.1996)
    and many more

Sweden          sunet.se              (existing feeds only!, see below)
Norway          news.uio.no           usenet@usenet.no (since XI 1996)
Netherlands     news.easy.nl          Peter Janssens <pj@easy.nl> (V 1996)
Netherlands	news.NL.net, news.xs4all.nl, news.IAEhv.nl   see note below
Germany         news.uni-stuttgart.de Kurt Jaeger <news@news.uni-stuttgart.de>
Germany         news.desy.de          Kars Ohrenberg <ohrenber@zedy00.desy.de>
Germany         news.fu-berlin.de	Vera Heinau et. al.
Australia       news.ozemail.com.au
Australia       news.mel.aone.net.au 

Poland          news.ict.pwr.wroc.pl  Tomasz Surmacz <tsurmacz@ict.pwr.wroc.pl>
Poland          news.uw.edu.pl        Michal Jankowski <michalj@fuw.edu.pl>
Poland          news.icm.edu.pl       Jan Rychter <jwr@news.icm.edu.pl>
Poland          news.nask.pl          admin account

    and all existing news servers in Poland (check
    http://www.usenet.pl/doc/ for the more up-to-date list)
Send e-mail to any of the above news administrators, asking him if he could feed you the pl.* hierarchy. Include your news server addresses: the internet address of the server and its 'news name' ie. the string which is put in the 'Path:' header. Unfortunately, sunet.se seems to be overloaded with its newsfeeds, so its administrators will not give you a 'new feed', but if you already have one - just ask for the pl.* hierarchy to be added to your existing feed. In Netherlands you should probably get the pl.* groups from your 'usual' neighbour. If not, contact admins of news.NL.net.

Otherwise, contact other people from this list.

In reply you should get appropriate host name/news name for the host you will be receiving pl.* articles from. (The first will be exactly as in the list above (let it be "sample.site.pl", but the 'Path:' address is usually a bit different - say "news.pl".) You should also get the current list of all pl.* groups that you ought to create manually. This list is also posted regularly to pl.answers and pl.news.admin groups (you may connect to news.ict.pwr.wroc.pl to get access to these groups before you establish the proper feed for pl.* for yourself). You can get the part of newsgroups and active file also via ftp, as stated below.

Add the internet address (sample.site.pl) to your hosts.nntp file and restart the server or do a "ctlinnd reload hosts.nntp '' ", if you are running INND. This way you start recognizing the other host as the news server and not a client. The 'PathHost' ("news.pl" in our example) will be needed for the second step - sending news back.

Now - create all the groups. You should get the list in the format of the active file, so this little script should do the job:

--------------------------- add_pl.sh ------------------------------
#!/bin/sh
# Usage: add_pl file_containing_pl_groups_in_active_format
# such as http://www.usenet.pl/doc/pl.active
# or ftp://ftp.ict.pwr.wroc.pl/pub/faq-pl/pl.active

# change this to reflect your configuration
ACTIVE=/usr/lib/news/active
PATH=$PATH:/usr/bin:/bin:/usr/lib/news/bin

# Pause the server  (assuming innd)
ctlinnd pause 'updating active' 

cp ${ACTIVE} ${ACTIVE}.bak
awk '{ printf ("%s 0000000000 0000000001 %s\n", $1, $4); }' < $1 >> $(ACTIVE)

ctlinnd reload active 'updating active' 
ctlinnd go 'updating active' 
----------------------------------------------------------------------
or if there are not too many groups, then (for INND only):
--------------------------- add_pl_small.sh --------------------------
#!/bin/sh
# Usage: add_pl_small file_containing_pl_groups_in_active_format
# such as http://www.usenet.pl/doc/pl.active

# change this to reflect your configuration
PATH=$PATH:/usr/bin:/bin:/usr/local/news/bin

while read group last first type ; do
        ctlinnd newgroup ${group} ${type}
done < $1
----------------------------------------------------------------------
You should also get the descriptions of all the pl.* groups so just 'cat >>' them to the end of your 'newsgroups' file. The current list of all pl.* groups (updated daily) can be downloaded via ftp from the following address:
    ftp://ftp.ict.pwr.wroc.pl/pub/faq-pl/pl.newsgroups
The portion of current 'active' file containing only pl.* groups can be found at:
    ftp://ftp.ict.pwr.wroc.pl/pub/faq-pl/pl.active
Even better and simpler way of creating new groups and getting in sync is getting just the pl.newsgroups file and feeding it directly to the docheckgroups program. This will produce a series of 'ctlinnd newgroup ... lines which can be fed to shell for execution (ie. 'cat pl.newsgroups | docheckgroups | sh')


moderators file:

All the postings to the moderated groups under pl.* should be sent to news-group-name@usenet.pl (with dots in the group name replaced by hyphens) - so to your 'moderators' file add the following somewhere near the beginning (before the default "*:%s@uunet.uu.net"):
    pl.*:%s@usenet.pl
If you have installed the innd1.5.1 server and config files, you just have to uncomment the appropriate line in the moderators file.

control.ctl file

To get all the new pl.* groups created and removed automatically when needed, add the following to the control.ctl file:
newgroup:newgroup@usenet.pl:pl.*:doit=newgroup
rmgroup:newgroup@usenet.pl:pl.*:doit=rmgroup
If you can check pgp-signed control messages, look at the end of this document to get more info about how to set it up for the pl.* hierarchy.

expire.ctl file

This is completely your (or your users) policy, how long to keep the postings in each group, but let me suggest you just one thing. There is a special group pl.answers which is much like all *.answers groups - it is moderated and all articles there are sent periodically with the appropriate 'Expires:' and 'Supersedes:' headers, so it would be best, if these were respected. If your 'standard' entry is like: '*:A:1:10:never' - ie. 'never' remove articles with 'Expires:' header before they expire - don't worry about pl.answers. But if you force all articles to be expired after say 10 days - please make an exception and add to the expire.ctl:
       pl.answers:A:1:10:never 
or:
       pl.answers:A:1:10:60
which means keep articles there for at least 1 day, usually 10 days if there is no 'Expires:' header, otherwise respect the header (but in the latter case - not longer than 60 days even if Expires: header says it should be kept longer). All the articles have maximum 30-40 days expiration period, so any of the above will be ok. (It is a good idea to respect these headers in other groups too, anyway).


2. Sending news

Unless you are receiving pl.* groups through your mainstream feed, you need to reconfigure a bit your server to send pl.* groups back. And even if you are receiving them through your normal feeds, please make sure they are included in the list of groups sent out.

When you asked for a pl.* feed, you also got permission to send all pl.* groups back, so to your 'newsfeeds' file (that's in innd; in cnews it is called 'sys') add appropriate entry for sending the pl.* groups back. Use the 'pathhost' name (in our example - "news.pl") that you have got, to build the entry. For example, in innd (assuming getting news from sample.site.pl and sending them back there) it might look like:

    plnews/news.pl\
        :!*,pl.*\
        :Tf,Wnm:
(replace "news.pl" with what appears in the Path: field put there by the site feeding you) and then you also need to tell your cron job sending news, to send them also to "sample.site.pl". If you are using nntpsend/innxmit, add to the nntpsend.ctl file:
    plnews:sample.site.pl:::-T1720 -t300
    ^      ^                ^
    |      |                + Other useful parameters
    |      Internet addres of your pl.* provider
    +-- Name used in newsfeeds file.
That should be all... You may now post a test message to pl.test and expect a reply in e-mail within a day or so, as there is an automatic pl.test responder attached to one of the servers in Poland. If your test posting reaches it - get yourself a beer, as you have done a good work. :-) If you do not want to get the reply message - just include the word 'ignore' in the message subject or body.


======================================================================

Once again, step by step, in short:

You probably know your own news system better than me, so these detailed descriptions may be unnecessary. If you know how to do most of things, use this short 'checklist' instead:

That's all.

Now is the time for...

Questions...

What are the pl.* groups anyway?

The pl.* news hierarchy holds Polish national groups. They sometimes mirror similar groups in the big-eight hierarchy (with the exception, that in pl.* groups the most used language is Polish), but some are very specific.

Is there any 'checkgroups' message for pl.* groups?

Yes, there is. It gets posted automatically once every two months (starting on January 1st), so you should not see it very frequently. In addition you may get the list of pl.* groups in the 'newsgroups' file format from the locations described above and feed it to the 'docheckgroups' program.

Should I be concerned about large volume of articles in binary sub-hierarchies?

No. There are no binary groups under the pl.* hierarchy and there are none planned. Any binary postings sent to pl.* are usually dealt with by friendly cancelbots.

What about PGP-signed control messages?

Yep, the control messages requesting creation or removal of groups in the pl.* hierarchy are already PGP-signed. Of course, if your server does not support checking of PGP signatures in control messages, you have to trust the 'From:' headers. innd versions 1.5.1 and later support PGP control messages by default, so you just have to uncomment appropriate lines in the control.ctl file, as they are commented-out by default. The public PGP key for these messages (ie. the key for pl.announce.newgroups) is included in the innd 1.5.1 config files and can be also found here:

pub  1024/838AF8ED 1996/05/31  pl.announce.newgroups
sig       838AF8ED               pl.announce.newgroups
                               
-----BEGIN PGP PUBLIC KEY BLOCK-----
Version: 2.6.2i

mQCNAzGvEY4AAAEEAL8clpC5yydhkMN1ppivfj3E6EIyZtntScjuCyI+RwpvLf9o
So+FLbiMmjjCV7BNjds7c1yaDMPDFetZlqEYKof5Q0QK2bDaaou3wtnDesAXqrde
K24GURn0N2BYWR9sEFJ4bwn8fudHelFcAXPt24lI29bu6dX+LddKTteDivjtAAUR
tBVwbC5hbm5vdW5jZS5uZXdncm91cHOJAJUDBRAyc5r/10pO14OK+O0BAWq9A/0e
So5/56x6hkT0to92FUiIbcoHF6J20sa4edDXbzTHZTvNJh3K4iSBwkspMcx8AWyS
36ckftXsZP2xRTj2Dk4xgLLCcFBjPHt7q5ysa8dChkgEu8770OE5atbQZorzz757
xpOgKmjRXLVbzt680Z807jH/zOo+BRby3ZMJFtGPyrQXPGNvbnRyb2xAYWRtLnVz
ZW5ldC5wbD4=
=n+Qy
-----END PGP PUBLIC KEY BLOCK-----
You may also send email to
pgp-public-keys@pgp.ai.mit.edu with "GET 0x838AF8ED" in the Subject: field and empty message body.

The corrected control.ctl lines should be as follows:

newgroup:newgroup@usenet.pl|michalj@*fuw.edu.pl:pl.*:verify-pl.announce.newgroups
rmgroup:newgroup@usenet.pl|michalj@*fuw.edu.pl:pl.*:verify-pl.announce.newgroups
They are not set as a default for innd 1.5.1, so you need to comment out 'old' newgroup/rmgroup files and copy the above ones.

For more information on checking PGP-signed control messages, see ftp://ftp.uu.net/networking/news/misc/pgpcontrol/README.html or ftp://ftp.uu.net/networking/news/misc/pgpcontrol/README.
Whether or not you do a PGP validation of control messages, the addresses they are sent from are the same as mentioned above (in the control.ctl section).


If you have any comments on the above procedure or my description, or if you find any errors, etc., please do tell me, so I can improve it.


Tomasz R. Surmacz (tsurmacz@adm.usenet.pl, tsurmacz@ict.pwr.wroc.pl), 22.11.2005

RCSid: $Id: news-pl-new-site-faq.html,v 2.6 2005/11/22 03:00:16 tsurmacz Exp $