I ran into a problem apt-mirror: can't open index apt.tn.ixsystems.com/apt-direct/bluefin/nightlies/nodejs//dists/bullseye/main/binary-i386/Packages in process_index at /usr/bin/apt-mirror line 934
using pseudolegging, I found out that the index variable already has a slash.

!!! path=apt.tn.ixsystems.com/apt-direct/bluefin/nightlies/nodejs
!!! mirror=/mnt/pool/apt/mirror/apt.tn.ixsystems.com/apt-direct/bluefin/nightlies/nodejs
!!! index=/dists/bullseye/main/binary-all/Packages

I slightly changed the code in line 908 by adding a filter to remove the slash at the beginning of the line. The error disappeared.
$index =~s/^\///;

Me again,

Looking to setup raspbian with apt-mirror rather than their own tool that is incredibly slow (talking 5 days in at this point) after downloading 1.4tb of other mirrors with apt-mirror in a few hours. Tried with the documented ways and faced connection refused from their mirrors, likely a me issue and with their poor documentation on getting a mirror running

This isn't an issue with the project obviously, and purely with my brain and familiarity with pulling "nonstandard" architecture.

Any help in getting this added and mirroring at literally 200 times the speed would be appreciated, haha.

Thanks!

After cloning this repo I replaced /usr/bin/apt-mirror with the cloned repo
and changed the permission using this command
sudo chown root:root /usr/bin/apt-mirror && sudo chmod 755 /usr/bin/apt-mirror
But as you can see in the screenshot, I am getting a permission denied error.
Please help me to fix it.
Thanks.

Downloaded 18.04 and 20.04 mirrors and seems I have everything I need, however when pointing a local or even remote server to my mirror, by-hash folders are missing.

Likely user error but I can't seem to find any up to date information on fixing it.

Cheers!

Hi Folks,
I'm just a user and not a developer by any means, so I may be getting this wrong, but it sort of seems like this fork is now actually behind some recent work on the upstream project.

A couple years ago I started using this fork because the upstream appeared to be abandoned and IIRC some compression scheme had changed in how repodata was compressed - but it now looks like there is some maintenance happening upstream.

If that's correct and I'm not mixed up, would it be possible to get this fork rebased to pull in any upstream enhancements or fixes?

Or since the original maintainer is still looking for maintainers and there is active work happening, maybe this fork could merge back with upstream and bring over whatever fixes or enhancements are present on the fork and maintenance could once again happen all on the same project?

Thanks, and apologies again if I've gotten any of this wrong.

Error:
"apt-mirror: can't open index mirror01.idc.hinet.net/ubuntu//dists/jammy-backports/universe/binary-amd64/Packages in process_index at /usr/bin/apt-mirror line 932.
]"

This is one of the many errors, but they look alike, so I just post one of them.

My guess is that there are double slashes for the url somewhere in the program.

I'm still getting this sort of error when I try to update a client system from my mirror (immediately after the mirror has updated)
Fallo al obtener http://xxx/mx.archive.ubuntu.com/ubuntu/dists/jammy-updates/main/dep11/icons-48x48.tar.gz File has unexpected size (24973 != 26204). Mirror sync in progress? [IP: 192.168.4.200 80]
Hashes of expected file:
- Filesize:26204 [weak]
- SHA256:7dc2c96df3d401ee2f3d904940c210e3d3a8a911a1e23d1f028929d1b6698ccd
- SHA1:372e5e432750a3adecba56e6a1b57c5117e43849 [weak]
- MD5Sum:744c5e9a709589c13bafe4d2e5b53e44 [weak]
Release file created at: Fri, 24 Jun 2022 23:40:28 +0000

I have this section in my apt-mirror mirror.list file:

Graylog

#deb https://packages.graylog2.org/repo/debian/ stable 4.2
deb https://packages.graylog2.org/repo/debian/ stable 4.3
deb https://packages.graylog2.org/repo/debian/ sidecar-stable 1.1
deb https://packages.graylog2.org/repo/debian/ sidecar-stable 1.2
clean https://packages.graylog2.org/

Now, having just added the versions stable 4.3 (for graylog-server package) and sidecar-stable 1.2 (for graylog sidecar), I see that apt-mirror has not managed to fetch the new versions, despite the fact that apt-mirror (latest version from your github repo) throws no errors when being run with this sources.list configuration.

This is proven by listing the contents of ie. the sidecar-stable repo on the apt-mirror box:

Am I doing something wrong, or can apt-mirror actually not handle mirroring multiple versions of the same piece of software?

eg http://ch.archive.ubuntu.com/ubuntu/dists/jammy-updates/main/binary-amd64/by-hash/

seems to be there for a while, eg:
apt-mirror#134

older workaround:
apt-mirror#58

maybe you have a solution ?

I copied the apt-mirror in /usr/bin followed by change in permissions. Then I ran this apt-mirror However, it showed errors like :

(a) Failed to open Release file from http://archive.ubuntu.com/ubuntu/dists/Release at /usr/bin/apt-mirror line 522,
(b) Failed to open Release file from http://archive.ubuntu.com/ubuntu/dists//Release at /usr/bin/apt-mirror line 669,
(c) apt-mirror: can't open index archive.ubuntu.com/ubuntu//dists//focal/binary-amd64/Packages in process_index at /usr/bin/apt-mirror line 817.

deb-armhf http://mirror.aarnet.edu.au/pub/raspbian/raspbian [ONE Separator] bullseye [Multiple Separators allowed] main firmware contrib non-free rpi

Then in this part of the line main firmware contrib non-free rpi it must only contain 1 separator between each item.

Between URL and repository name MUST BE ONE Separator.
Between repository name and sections can be multiple Separators, spaces/tabs (but not both).
Between each section being mirrored, MUST BE ONE Separator.

Does this need fixing? Or does this just need documentation/clarification in the conf file?

Seems like #8 was identifying this problem, compounded by spreading sections across lines (not expected, and most likely unhandled by code).

I'm getting a strange behaviour when running apt-mirror (as apt-mirror user, with: sudo -u apt-mirror apt-mirror).

At first, things seem normal, as I get this output:

Downloading 286 index files using 1 threads...
Begin time: Wed Oct 20 12:40:08 2021
[1]... [0]...
End time: Wed Oct 20 12:40:24 2021

Processing translation indexes: [TTTTTTTTTTTT]

Downloading 47 translation files using 1 threads...
Begin time: Wed Oct 20 12:40:24 2021
[1]... [0]...
End time: Wed Oct 20 12:40:25 2021

Processing DEP-11 indexes: [DDDDDDDDDDDD]

Downloading 15 dep11 files using 1 threads...
Begin time: Wed Oct 20 12:40:25 2021
[1]... [0]...
End time: Wed Oct 20 12:40:25 2021

Then things go sideways, as I get well over a thousand of these error messages (seems only difference between each error is the line number at the end):

Processing indexes: [SSUse of uninitialized value $lines{"Files:"} in split at /usr/bin/apt-mirror line 846, <STREAM> line 1.
Use of uninitialized value $lines{"Files:"} in split at /usr/bin/apt-mirror line 846, <STREAM> line 2.
Use of uninitialized value $lines{"Files:"} in split at /usr/bin/apt-mirror line 846, <STREAM> line 3.
Use of uninitialized value $lines{"Files:"} in split at /usr/bin/apt-mirror line 846, <STREAM> line 4.
...

The application does manage to download from repositories despite the errors - this is the output I get when it finishes up:

Use of uninitialized value $lines{"Files:"} in split at /usr/bin/apt-mirror line 846, <STREAM> line 4.
Use of uninitialized value $lines{"Files:"} in split at /usr/bin/apt-mirror line 846, <STREAM> line 5.
Use of uninitialized value $lines{"Files:"} in split at /usr/bin/apt-mirror line 846, <STREAM> line 6.
SSSPPPPPPPPPPPP]

327.1 MiB will be downloaded into archive.
Downloading 1 archive files using 1 threads...
Begin time: Wed Oct 20 12:40:31 2021
[1]... [0]...
End time: Wed Oct 20 12:40:31 2021

50.1 MiB in 44 files and 0 directories can be freed.
Run /data/debian/var/clean.sh for this purpose.

Running the Post Mirror script ...
(/data/debian/var/postmirror.sh)


Post Mirror script has completed. See above output for any possible errors.

For some reason, I'm unable to mirror the repo https://packages.sury.org/php

When I add that repo to my mirror.list like so:
deb https://packages.sury.org/php bullseye main

I get this when running apt-mirror:

What might be the issue here?

Using that exact same repo directly on a client's sources.list works just fine:

when I attempt to update a client from the mirror:
Err:5 http://xxx/security.ubuntu.com/ubuntu jammy-security/main amd64 Packages
File has unexpected size (191412 != 191800). Mirror sync in progress? [IP: 192.168.4.200 80]
Hashes of expected file:

• Filesize:191800 [weak]
• SHA256:c0934666b61e4112778569d1f8f0f5613779dabb1f524fc324b65b2722c304f8
• SHA1:c70e0339ff96a09f0503101b6497995adea2de53 [weak]
• MD5Sum:b6ee6a88710edc5e65d1aba3152e9507 [weak]
  Release file created at: Wed, 29 Jun 2022 22:22:42 +0000
  Des:14 http://xxx/security.ubuntu.com/ubuntu jammy-security/main Translation-en [46.5 kB]

Hi guys, great that you made this updated version, and I think it can be improved

I just lost 1TB of mirror data ( not only Ubuntu for amd64 ), when replaced the original apt-mirror Ubuntu 20.04 with your version ( no hard feelings, its not unique data, just takes some waiting time :-)

Same issue as #5

Using just 1 line as test :
deb http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse
leads to :
Failed to open Release file from http://archive.ubuntu.com/ubuntu/dists/Release at /usr/bin/apt-mirror line 522.

Details
I used a nice layout in the list file to keep the overview :

deb-amd64 http://archive.ubuntu.com/ubuntu focal main universe
deb-amd64 http://archive.ubuntu.com/ubuntu focal-security main universe
deb-amd64 http://archive.ubuntu.com/ubuntu focal-updates main universe

deb-i386 http://archive.ubuntu.com/ubuntu focal main universe
deb-i386 http://archive.ubuntu.com/ubuntu focal-security main universe
deb-i386 http://archive.ubuntu.com/ubuntu focal-updates main universe

Your tool does not accept the many spaces, and then does a cleanup, effectively deleting all existing data from my disk, so not really an 1:1 replacement of the original (yet)

When I use
deb http://archive.ubuntu.com/ubuntu focal main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu focal-updates main restricted universe multiverse

it does work perfectly for one machine I tested

So perhaps you can change the tool to accept many spaces in the list lines ?
Or add a warning on the front page ?

did not yet test amd64/i386 to limit the size, as I think I now have all architectures, right ?

Thanks, keep up the good work 👍

apt-mirror: can't copy /media/ubuntu-repo/22.04/skel/archive.ubuntu.com/ubuntu/dists/jammy-updates/Contents-i386.gz to /media/ubuntu-repo/22.04/mirror/archive.ubuntu.com/ubuntu/dists/jammy-updates/Contents-i386.gz at /usr/bin/apt-mirror line 929.

Hi Stifler
(no idea why parts of the issue are larger fonts, not my intention )

After removing ( by a script, not manually anymore :-) ) the tabs and double spaces, your tool has been performing perfectly, Thanks

Now migrating to debian-bullseye and Ubuntu-jammy

All repos for ubuntu jammy / raspbian bullseye mirror perfectly for amd64 / armhf / arm64

Only docker / jammy for armhf and arm64 fail

Hopefully you can solve that ?

=================================================================================
Details :

Sites look the same to me. e.g.
https://download.docker.com/linux/ubuntu/dists/jammy/stable/binary-armhf/
https://download.docker.com/linux/ubuntu/dists/focal/stable/binary-armhf/

Could not quickly find a version number in the script, I think that would help.
I am using the version I downloaded 2022-07-27

Errors:
Only thing that looks strange is the double // in ubuntu//dists, see error lines below, no idea why it is there or if that is normal

=================================================================================
armhf

deb-armhf https://download.docker.com/linux/raspbian/ buster stable
ok

deb-armhf https://download.docker.com/linux/raspbian/ bullseye stable
ok

deb-armhf http://download.docker.com/linux/ubuntu focal stable
ok

deb-armhf http://download.docker.com/linux/ubuntu jammy stable
apt-mirror: can't open index download.docker.com/linux/ubuntu//dists/jammy/stable/binary-armhf/Packages in process_index at /usr/bin/apt-mirror line 932.

=================================================================================
arm64

deb-arm64 https://download.docker.com/linux/ubuntu focal stable
OK

deb-arm64 https://download.docker.com/linux/ubuntu jammy stable
apt-mirror: can't open index download.docker.com/linux/ubuntu//dists/jammy/stable/binary-arm64/Packages in process_index at /usr/bin/apt-mirror line 932.

=================================================================================

Dear all,

Is it possible to exclude some packages with apt-mirror? For example, i don't want to download games, and many gnome/kde packages, etc. This can save a lot disk space for a local apt repo.