I would like to build my own local repository on my LAN, so that machines on the LAN can update and upgrade from it. I want to download the packages and store them on my local server so that I can update, upgrade, install, etc, from it without using the internet.
-
2Possible Duplicate:askubuntu.com/questions/974/…– stephenmyallCommented Jul 31, 2012 at 9:40
-
4I don't think it is a duplicate. What maythux wants to accomplish is create his own repository server for use with aptitude. What Keryx does is replace aptitude as package manager and create external sources for packages.– con-f-useCommented Jul 31, 2012 at 10:57
-
1Possible duplicate? - askubuntu.com/questions/9809/… or askubuntu.com/questions/3503/…– jrgCommented Jul 31, 2012 at 13:42
-
1Since Ubuntu 22 (20?) repositories must be signed. These answers are now all obsolete and incomplete because they do not include that part.– user10489Commented Jun 9 at 3:41
10 Answers
From the Ubuntu Help wiki:
There are 4 steps to setting up a simple repository for yourself
- Install
dpkg-dev
- Put the packages in a directory
- Create a script that will scan the packages and create a file
apt-get update
can read- Add a line to your
sources.list
pointing at your repositoryInstall dpkg-dev
Type in a terminal
sudo apt-get install dpkg-dev
The Directory
Create a directory where you will keep your packages. For this example, we'll use
/usr/local/mydebs
.sudo mkdir -p /usr/local/mydebs
Now move your packages into the directory you've just created.
Previously downloaded Packages are generally stored on your system in the
/var/cache/apt/archives
directory. If you have installedapt-cacher
you will have additional packages stored in its/packages
directory.The Script
update-mydebs
It's a simple three-liner:
#! /bin/bash cd /usr/local/mydebs dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
Cut and paste the above into gedit, and save it as
update-mydebs
in~/bin
. (The tilde '~' means your home directory. If~/bin
does not exist, create it: Ubuntu will put that directory in yourPATH
. It's a good place to put personal scripts). Next, make the script executable:chmod u+x ~/bin/update-mydebs
How the script works:
dpkg-scanpackages
looks at all the packages inmydebs
, and the output is compressed and written to a file (Packages.gz
) thatapt-get update
can read (see below for a reference that explains this in excruciating detail)./dev/null
is an empty file; it is a substitute for an override file which holds some additional information about the packages, which in this case is not really needed. See deb-override(5) if you want to know about it.Sources.list
add the line
deb file:/usr/local/mydebs ./
to your
/etc/apt/sources.list
, and you're done.CD Option
You can burn the directory containing the debs to a CD and use that as a repository as well (good for sharing between computers). To use the CD as a repository, simply run
sudo apt-cdrom add
Using the Repository
Whenever you put a new deb in the mydebs directory, run
sudo update-mydebs sudo apt-get update
Now your local packages can be manipulated with Synaptic, aptitude and the apt commands:
apt-get
,apt-cache
, etc. When you attempt toapt-get install
, any dependencies will be resolved for you, as long as they can be met.Badly made packages will probably fail, but you won't have endured
dpkg
hell.
-
4Could you explain the syntax on the line
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
. What's/dev/null
doing there. I read the man page too, but it wasn't quite clear. Commented May 28, 2013 at 12:13 -
@blade19899 I need a small bit of clarification, please. I want a repository with just a few select packages in it, not every package I ever touched. Am I correct that this technique will give me that ability? The goal here is to have a repository that a software installation group can use on an isolated LAN, far away from the temptations to apt-get the unneeded. Commented Jun 28, 2013 at 19:24
-
-
-
18Note that if you use
deb file:/usr/local/mydebs ./
in your/etc/apt/sources.list
, you may face the problem E: The repository 'file:/home/path Release' does not have a Release file. N: Updating from such a repository can't be done securely, and is therefore disabled by default. To solve the problem usedeb [trusted=yes] file:/usr/local/mydebs ./
. Commented Mar 19, 2019 at 14:26
*To make an offline Repository Over LAN *
Install a Local Apache Webserver
# apt-get install apache2
By default, Debian's Apache package will set up a website under /var/www
on your system. For our purposes, that's fine, so there's no reason to do anything more. You can easily test it by pointing your favorite browser at http://localhost
You should see the default post-installation web page which is actually stored in /var/www/index.html
Create a Debian Package Repository Directory
chose to create a directory /var/www/debs
for this. Under it, you should create "architecture" directories, one for each architecture you need to support. If you're using just one computer (or type of computer), then you'll only need one -- typically "i386" for 32-bit systems or "amd64" for 64 bit. If you are using some other architecture, I'll assume you probably already know about this.
Now just copy the ".deb" package files for a given architecture into the appropriate directories. If you now point your favorite web browser at http://localhost/debs/amd64
(for example) you'll see a listing of the packages for 64 bit systems.
Create a Packages.gz file
Now we need to create a catalog file for APT to use. This is done with a utility called "dpkg-scanpackages". Here's the commands I use to update the AMD64 packages on my LAN:
# cd /var/www/debs/
# dpkg-scanpackages amd64 | gzip -9c > amd64/Packages.gz
Make the repository known to APT
Now the only thing left to do is to let APT know about your repository. You do this by updating your /etc/apt/sources.list file. You'll need an entry like this one:
deb http://localhost/debs/ amd64/
I used the actual hostname of my system instead of localhost -- this way the code is the same for all of the computers on my LAN, but localhost will do just fine if you are running just one computer.
Now, update APT:
# apt-get update
-
2Adding that line to /etc/apt/sources.list will break updates when not in the LAN, won't it?– FelixCommented Dec 3, 2015 at 14:43
-
2For Ubuntu 16.04, you might need to replace
/var/www/debs
in this answer with/var/www/html/debs
. Or you will need extra steps to manually edit your apache configuration in/etc/apache2
Commented Feb 18, 2018 at 22:06
Creating an Authenticated Repository
I've had a look at the answers here and on other sites and most have the (IMHO big) disadvantage that you're setting up an unauthenticated repository. This means you need to run apt-get
with --allow-unauthenticated
to install packages from it. This can be a security risk, especially in scripts where the packages you're installing might not all be from your local repository.
Note that I haven't covered here how to make it available over the LAN, but that's fairly generic config using Apache or nginx (see the other answers here).
Setup the repo directory
mkdir /home/srv/packages/local-xenial
cd /home/srv/packages/local-xenial
Then add a line like this to sources.list
:
deb file:/home/srv/packages/local-xenial/ ./
Adding and Removing Packages
remove packages
rm /home/srv/packages/local-xenial/some_package_idont_like
add packages
cp /some/dir/apackage.deb /home/srv/packages/local-xenial
now run the following script which generates the Packages, Release and InRelease files and signs them with your gpg private key:
#!/bin/bash
# setup a simple apt-compatible repository having one component 'main' in a directory using most of the recommended the directory structure from Debian's official wiki (https://wiki.debian.org/DebianRepository/Format)
function rebuild_local_repo
{
DISTROCODENAME=${1} # E.g noble if using Ubuntu Noble 24.04, also called 'Suite' in the Debian Wiki
ARCHTOBUILDFOR=${2} # E.g amd64, arm64
COMPONENTNAME='main' # Most repos use 'main' or 'stable', probably doesn't matter what it is but it's good to be consistent
REPODIRECTORY="/srv/packages/localrepos/local-${DISTROCODENAME}"
if [ ! -d ${REPODIRECTORY} ]; then
echo "ERROR: repository directory ${REPODIRECTORY} doesn't exist"
exit 1
fi
cd ${REPODIRECTORY}
# Setup the required directories (https://wiki.debian.org/DebianRepository/Format)
install -d -p -o tim -g tim -m 750 dists/${DISTROCODENAME}
install -d -p -o tim -g tim -m 750 dists/${DISTROCODENAME}/${COMPONENTNAME}
install -d -p -o tim -g tim -m 750 dists/${DISTROCODENAME}/${COMPONENTNAME}/binary-${ARCHTOBUILDFOR}
RELEASEFILEPATH=dists/${DISTROCODENAME}/Release
INRELEASEFILEPATH=dists/${DISTROCODENAME}/InRelease
PACKAGESFILEPATH=dists/${DISTROCODENAME}/${COMPONENTNAME}/binary-${ARCHTOBUILDFOR}/Packages
# Generate the Packages file
dpkg-scanpackages . /dev/null > ${PACKAGESFILEPATH}
gzip --keep --force -9 ${PACKAGESFILEPATH}
# Generate the Release and InRelease files
# Required header for the release file:
RELEASEFILEHEADER="Origin: Tim_Local_Repo
Label: My_Local_Repo
Codename: ${DISTROCODENAME}
Architectures: amd64 arm64 i386
Components: main
Description: My local APT repository
SignWith: 12345DEF"
echo "$RELEASEFILEHEADER" > ${RELEASEFILEPATH} # note that to use multi-line vars in bash and preserve newlines you have to use "$VAR" not ${VAR}
# The Date: field has the same format as the Debian package changelog entries
echo -e "Date: `LANG=C date --utc -R`" >> ${RELEASEFILEPATH}
# Release must contain MD5 sums of all repository files (in a simple repo just the Packages and Packages.gz files)
echo -e 'MD5Sum:' >> ${RELEASEFILEPATH}
printf ' '$(md5sum ${PACKAGESFILEPATH}.gz | cut --delimiter=' ' --fields=1)' %16d '${COMPONENTNAME}/binary-${ARCHTOBUILDFOR}/Packages.gz $(wc --bytes ${PACKAGESFILEPATH}.gz | cut --delimiter=' ' --fields=1) >> ${RELEASEFILEPATH}
printf '\n '$(md5sum ${PACKAGESFILEPATH} | cut --delimiter=' ' --fields=1)' %16d '${COMPONENTNAME}/binary-${ARCHTOBUILDFOR}/Packages $(wc --bytes ${PACKAGESFILEPATH} | cut --delimiter=' ' --fields=1) >> ${RELEASEFILEPATH}
# Release must contain SHA256 sums of all repository files (in a simple repo just the Packages and Packages.gz files)
echo -e '\nSHA256:' >> ${RELEASEFILEPATH}
printf ' '$(sha256sum ${PACKAGESFILEPATH}.gz | cut --delimiter=' ' --fields=1)' %16d '${COMPONENTNAME}/binary-${ARCHTOBUILDFOR}/Packages.gz $(wc --bytes ${PACKAGESFILEPATH}.gz | cut --delimiter=' ' --fields=1) >> ${RELEASEFILEPATH}
printf '\n '$(sha256sum ${PACKAGESFILEPATH} | cut --delimiter=' ' --fields=1)' %16d '${COMPONENTNAME}/binary-${ARCHTOBUILDFOR}/Packages $(wc --bytes ${PACKAGESFILEPATH} | cut --delimiter=' ' --fields=1) >> ${RELEASEFILEPATH}
# Clearsign the Release file (that is, sign it without encrypting it)
gpg --yes --clearsign --digest-algo SHA512 --local-user tim -o ${INRELEASEFILEPATH} ${RELEASEFILEPATH}
# Release.gpg only need for older apt versions
# gpg -abs --digest-algo SHA512 --local-user tim -o Release.gpg Release
# Fix permissions
chmod o+rX ${REPODIRECTORY} -R
# Get apt to see the changes, the --allow-releaseinfo-change is in case anything in the RELEASEFILEHEADER has changed
sudo apt-get update --allow-releaseinfo-change
}
if [[ ${1} == '-h' || ${1} == '--help' ]]; then
echo -e "usage: `basename $0` DISTRO ARCH
where DISTRO is the Ubuntu version codename (e.g. 14.04 is trusty) and ARCH is the CPU architecture (amd64, arm64, i386)\n
The way to use this script is to do the changes to the repo first, i.e. delete or copy in the .deb file to /srv/packages/localrepos/local-DISTRO, and then run this script\n
This script can be run as the tim user (root is not needed)"
exit 0
fi
if [ "$#" -ne 2 ]; then
# if user didn't specify a distro version so try to update the repo for the current distro version
DISTROCODENAME=`lsb_release -c | awk '{print $2}'`
# if the user didn't specify the architecture update the repo for the current architecture
arch=$(uname -i)
ARCHITECTURE='amd64' # default to the most common arch
if [[ $arch == x86_64* ]]; then
ARCHITECTURE='amd64'
elif [[ $arch == i*86 ]]; then
ARCHITECTURE='i386'
elif [[ $arch == arm* ]]; then
ARCHITECTURE='arm64'
fi
echo -e "\n\nDefaulting to rebuilding the local repo for the current distro version, which is $DISTROCODENAME on $ARCHITECTURE\n"
fi
rebuild_local_repo "$DISTROCODENAME" "$ARCHITECTURE"
Links
https://wiki.debian.org/RepositoryFormat
http://ubuntuforums.org/showthread.php?t=1090731
https://help.ubuntu.com/community/CreateAuthenticatedRepository
-
@Phillip your edit used
date -Rc
, I corrected it todate -Ru
assuming that's what you meant from the edit description– muruCommented Jan 30, 2017 at 11:42 -
Thanks, I only recently started getting warnings from apt about this due to the date generated being in the local TZ and not UTC. I fixed it in my own script but forget to edit it here Commented Jan 30, 2017 at 22:14
-
1@KevinJohnson I've updated the main answer now with an example of that file from my local apt repo Commented Nov 4, 2017 at 2:36
The instructions in @BigSack's answer and Ubuntu's official wiki post didn't work for me on Ubuntu 18.04, until I made these two changes:
Generate a plain, uncompressed
Packages
file (when executing this, the working directory must be where all packages are located)cd /usr/local/mydebs dpkg-scanpackages -m . > Packages
Add the following entry in
/etc/apt/sources.list
deb [trusted=yes] file:/usr/local/mydebs ./
You can also setup local source server by nginx and reprepro:
Install debian packages
sudo apt-get install reprepro nginx
make directories for reprepro and edit it
sudo mkdir -p /srv/reprepro/ubuntu/{conf,dists,incoming,indices,logs,pool,project,tmp} $ cd /srv/reprepro/ubuntu/ $ sudo chown -R `whoami` . # changes the repository owner to the current user
/srv/reprepro/ubuntu/conf/distributions
Origin: Your Name Label: Your repository name Codename: karmic Architectures: i386 amd64 source Components: main Description: Description of repository you are creating SignWith: YOUR-KEY-ID
/srv/reprepro/ubuntu/conf/options
ask-passphrase basedir .
Include it in reprepro, build it
$ reprepro includedeb karmic /path/to/my-package_0.1-1.deb \ # change /path/to/my-package_0.1-1.deb to the path to your package
Config nginx:
/etc/nginx/sites-available/vhost-packages.conf
server { listen 80; server_name packages.internal; access_log /var/log/nginx/packages-access.log; error_log /var/log/nginx/packages-error.log; location / { root /srv/reprepro; index index.html; } location ~ /(.*)/conf { deny all; } location ~ /(.*)/db { deny all; } }
Optimize bucket size:
/etc/nginx/conf.d/server_names_hash_bucket_size.conf
server_names_hash_bucket_size 64;
Reference to Install Guide Link
-
4Whilst this may theoretically answer the question, it would be preferable to include the essential parts of the answer here, and provide the link for reference. Commented Jul 3, 2013 at 9:35
-
@elprup: you must have forgotten to update that answer :) Commented May 18, 2014 at 2:12
-
reprepor does not support multiple versions of the same package. It sounds strange, but this is the way reprepro works– maxadamoCommented Jun 19, 2019 at 9:01
There are several reasons you may want to create a local repository. The first is that you want to save on bandwidth if you have multiple Ubuntu machines to update. For example if you had 25 Ubuntu machines that all needed updating at least once a week, you would significantly save bandwidth because you could do all but the repository locally.
Most organizations have decent bandwidth for their network gateways but this bandwidth is a precious commodity that needs to be used wisely.
Many organizations still have routers with 10MB or 100MB limits at the gateway but 1 GB network connections internally so bandwidth could be better used internally. The second reason for creating your own repository is that you can control what applications are loaded on your internal Ubuntu machines.
You can remove any applications your organization does not want to use on the local network from the repository that updates the machines. Even better, you can create a test box and test applications and versions before you allow them to roll out into your network assuring security and stability.
You first have to setup a mirror, to do that you need to Just press Ctrl+Alt+T on your keyboard to open Terminal. When it opens, run the command below.
apt-get install apt-mirror
Once you have your set up apt-mirror you can start your download of the repository with this command.
apt-mirror /etc/apt/mirror.list1
1Source:Create an Ubuntu Repository
To make an offline local Repository
1. make a dir accessible (atleast by root)
sudo mkdir /var/my-local-repo
- copy all the deb files to this directory.
- scan the directory
sudo dpkg-scanpackages /var/my-local-repo /dev/null > /var/my-local-repo/Packages
- add the local repository to sources
echo "deb file:/var/my-local-repo ./" > /tmp/my-local.list
sudo mv /tmp/my-local.list /etc/apt/sources.list.d/my-local.list
sudo apt-get update
-
The more or less same thing is also on official wiki: Repositories/Personal - Community Help Wiki– sdaauCommented Feb 23, 2014 at 16:19
I tried to use apt-rdepends
like in the selected answer, but when I tried to install the package from my local repository, it complained about missing dependencies.
apt-rdepends
wasn't listing some of the dependencies for my package. I suspect it has something to do with the fact, that apt-cache show
shows multiple records for it.
Instead I used apt-cache depends
, and that did the trick:
Getting a recursive list of dependencies
apt-cache depends <packagename> -i --recurse
-i
: important dependencies only
--recurse
: recursive
Turn it into a digestible list
- Removing symbols & spaces:
| tr -d "|,<,>, "
- Removing Depends: & PreDepends:
| sed -e 's/^Depends://g' | sed -e 's/^PreDepends://g'
- Sorting the list:
| sort
- Only unique values:
| uniq > list.txt
Complete command:
apt-cache depends <packagename> -i --recurse | tr -d "|,<,>, " | sed -e \
's/^Depends://g' | sed -e 's/^PreDepends://g' | sort | uniq > list.txt
Download the packages
for i in $( cat list.txt ); do apt-get download $i; done;
Scan for the packages and turn it into Packages.gz
dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
-
1Might be a good idea to reference which answer you're talking about... Commented Jun 8, 2016 at 14:03
How to create CD/DVD .ISO repository?
The following is a script I had created (in Ubuntu 16.04 Xenial xerus) to create .ISO archive of .deb packages I had collected over time. I used to save a copy of every .deb packages downloaded, installed, system updates, (every .deb in /var/cache/apt/archives
before they disappeared, if they did) etc. so that my collection had every packages I'd ever need in an install of Ubuntu version, including their dependencies.
This script can be used to create a directory structure and necessary files to be able to use the .iso file as an APT repository. This script, however, does not create the ISO image, and tools such as xfburn
may be used. More help on what to copy to the root of the ISO image can be obtained in the script help.
You can save the following script as: deb_cache_pkg_to_repository_image_v0.1.sh .
#!/bin/bash
## v0.1
# This script does nothing more than easing the following few notable steps:
# ## Change to the directory where the package archives are cached:
# cd /usr/local/deb_packages
# ## Generate 'Packages' file and compress it for `apt':
# dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz
declare __SCRIPT_NAME="${0##*/}"
# declare __SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
declare __SCRIPT_INITIAL_WDIR="${PWD}"
declare __SCRIPT_ACTION=""
declare -a __DIR_REPOSITORY
declare __DIR_BASE="" ## The BASE_DIRECTORY directory
declare __LABEL_CODENAME="" ## The UBUNTU_CODENAME directory
declare __LABEL_BINARY_DIR="binary-i386"
declare __LABEL_SOURCE_DIR="sources_archive"
declare __PACKAGE_DETAIL_FILE="Packages"
declare __PATH_DEFAULT_BASENAME="/tmp/prepare_apt_archive_for_ISO"
declare __DEFAULT_README=""
declare -i __INDEX=0
declare __DIR_TMP="/tmp/.${__SCRIPT_NAME%.sh}"
declare __TMP_PKG_DETAIL="${__DIR_TMP}/Packages"
declare __LOG_DPKG_SCANPKG_MESSAGES="${__DIR_TMP}/dpkg_scanpackages_message.log"
declare __TMP_PRELIM_DEB_LIST="${__DIR_TMP}/all_required_debs_path.list"
declare __TMP_SUFFICIENT_DEB_LIST="${__DIR_TMP}/sufficient_required_debs_path.list"
declare __LOG_SCANNED_DIRS="${__DIR_TMP}/dpkg_scanpackages_directories.log"
function _Msg_Err() {
echo "${@}" >&2
}
function _Show_Help() {
while read -r ; do
echo "${REPLY}"
done <<-__EOF
SYNOPSIS: ${__SCRIPT_NAME} [OPTIONS] ACTION DIRECTORY_1 DIRECTORY_2 ...
where DIRECTORY_1 DIRECTORY_2 ... DIRECTORY_N are the directory(ies) you
must specify to scan for .deb packages. e.g. '/var/cache/apt/archives/' or
the directories where you have downloaded all those.
ACTION:
prepare
Creates the (temporary) directory structure, copies the required (from
among those available, as reported by \`dpkg-scanpackages') packages,
and generates proper 'Packages' and 'Packages.gz' files (containing the
package details). (This is the command you want to use initially; this
scans the enlisted directories with \`dpkg-scanpackages' and copies the
required ones into the BASE_DIRECTORY and then scans the .deb files to
prepare the 'Packages.xz' file.)
list
Lists out the package names, the latest ones, which have to be archived
ignoring the old packages. (Getting a "complete list" here, and for
preparing for backup is dependent on whether the specified repositories
as DIRECTORY_1, DIRECTORY_2, DIRECTORY_N contain packages with all their
Debian dependencies. This script does not check or resolve any of the
"dependencies unmet" issues and assumes complete repositories are
maintained in DIRECTORY_N : a limitation of this script.)
OPTIONS:
-D <BASE_DIRECTORY> | -
Use this to specify the base <BASE_DIRECTORY> where archive structure
will be created. e.g.: /tmp/prepare_apt_archive_for_ISO and also make
sure that the disk volume has enough space to copy the packages to
archive temporary. The default path '${__PATH_DEFAULT_BASENAME}'
will be used if the argument is a dash.
This structure will not be deleted by the script and must be done by
hand after you're done with creating a .iso image of \`apt' readable
package repository.
The contents of this directory should be at the DVD root; the contents
in this directory are the ones you will import to, say 'Xfburn' DVD
archive project, and create the ISO image.
-l <UBUNTU_CODENAME>
The label (name) for the directory <UBUNTU_CODENAME> which is to be
placed at the root of the ISO image. See ARCHIVE STRUCTURE for more.
-h
Show this help and exit.
Example usage:
$ ./deb_cache_pkg_to_repository_image_v0.1.sh -D - \\
-l xenial_xubuntu1604 \\
prepare /var/cache/apt/archives/ /opt/backup_deb_repo/
ARCHIVE STRUCTURE:
\$BASE_DIRECTORY/
|
+-- README.info
|
+-- \$UBUNTU_CODENAME/
|
+-- binary-i386/
| |
| +-- ( *.deb )
|
+-- Packages
+-- Packages.gz
|
+-- sources_archive/
|
+-- ( sources_of_packages_in_respective_directories )
In the above archive structure, the \$BASE_DIRECTORY can be any temporary
directory which will hold the the structure that begins at \$UBUNTU_CODENAME
directory which may be the codename of the Ubuntu release for which the
packages (binary and sources) are being archived or the name of the distro
with release number. e.g.: "bionic" or "xubuntu16.04.4" which will be at
the root / of the ISO image which may be created with tools like Xfburn.
If there are any README.info or other similar help documents, those should
be manually copied to this directory.
This script will "copy" all the required Debian package archives (as
determined by \`dpkg-scanpackages' in the 'binary/' or 'binary-i386/'
directory. Proper 'Packages' and 'Packages.gz' files containing details of
each packages in the 'binary/' directory will be placed alongside in the
\$UBUNTU_CODENAME directory. Also, a sub-directory named 'sources_archive/'
may be created to hold the source code archives of application packages
(and their dependent source) in the respective directories.
Using the CD/DVD media thus created:
ALTERNATIVE 01:
Add the following line to your '/etc/apt/sources.list' file:
deb file:/iso_mount_point ./
where '/iso_mount_point' is the directory where the ISO is mounted.
( Tested again, twice. This works fine. )
## deb file:/${__LABEL_CODENAME} ${__LABEL_BINARY_DIR}
## or
## deb file:/${__LABEL_CODENAME}/${__LABEL_BINARY_DIR} ./
## whichever works (note: the first method has been verified to be
## working).
ALTERNATIVE 02:
If the created ISO is compatible to be used with \`apt-cdrom', the following
steps are required to use the archived repository on the CD/DVD:
01. Burn to a CD/DVD; (or else, if you could, mount the ISO image);
02. Edit the '/etc/fstab' file to add the following line:
/dev/cdrom /media/cdrom iso9660 ro 0 0
where '/dev/cdrom' is the device file, '/media/cdrom' is the mount
point 'iso9660' the filesystem of the CD (udf if DVD) to mount
with 'ro' mount option, use zeros for the last two. (Sometimes, I
could specify .iso image path in place of '/dev/cdrom' and suggest
the mount point at any path in place of '/media/cdrom'. For detail,
on the syntax, consult the help page for \`mount' command or the
'fstab' file.
03. Run the command:
apt-cdrom add
04. Now, you may use \`apt' to update and install from the CD/DVD:
apt-get update
Note: After you add the repository (esp. on an offline computer), and
perform a \`apt-get update', the subsequent querries (in offline system)
were limited to the packages available in the local repositories; which
means you will not be able to search or view discriptions of any package
beyound the locally archived ones. (Find a way to backup those!!)
ALTERNATIVE 03:
Read the solution in the link: https://askubuntu.com/a/1501055/212123
AUTHOR: Avadhesh J. Thapa
__EOF
}
function _Write_Default_Readme() {
## Requires: __DEFAULT_README , __DIR_BASE variables to initialized
## and respective files/directories created.
## Make sure these are satisfied before this function gets called.
if command touch "${__DEFAULT_README}" ; then
> "${__DEFAULT_README}"
while read -r ; do
echo "${REPLY}" >> "${__DEFAULT_README}"
done <<-__EOF
Using this CD/DVD media:
ALTERNATIVE 01:
Add the following line to your '/etc/apt/sources.list' file:
deb file:/iso_mount_point ./
where '/iso_mount_point' is the directory where the ISO is mounted.
( Tested again, twice. This works fine. )
## deb file:/${__LABEL_CODENAME} ${__LABEL_BINARY_DIR}
## or
## deb file:/${__LABEL_CODENAME}/${__LABEL_BINARY_DIR} ./
## whichever works (note: the first method has been verified to be
## working).
ALTERNATIVE 02:
If the ISO created is compatible to be used with \`apt-cdrom', the following
steps are required to use the archived repository on the CD/DVD:
01. Burn to a CD/DVD; (or else, if you could, mount the ISO image);
02. Edit the '/etc/fstab' file to add the following line:
/dev/cdrom /media/cdrom iso9660 ro 0 0
where '/dev/cdrom' is the device file, '/media/cdrom' is the mount
point 'iso9660' the filesystem of the CD (udf if DVD) to mount
with 'ro' mount option, use zeros for the last two. (Sometimes, I
could specify .iso image path in place of '/dev/cdrom' and suggest
the mount point at any path in place of '/media/cdrom'. For detail,
on the syntax, consult the help page for \`mount' command or the
'fstab' file.
03. Run the command:
apt-cdrom add
04. Now, you may use \`apt' to update and install from the CD/DVD:
apt-get update
ALTERNATIVE 03:
Read the solution in the link: https://askubuntu.com/a/1501055/212123
__EOF
fi
}
function _Scan_Packages_Temp() {
## Input: name reference to the array variable containing directories to scan
## Output: outputs a few files of which the main required file is
## "${__TMP_SUFFICIENT_DEB_LIST}" containing the path of all .deb packages
## that should be copied to the archiving directory.
declare -n deb_repos=${1} ## Reference of array variable containing
## directories of Debian archives
declare -i i=0
## Clear the contents of temporary Packages detail list and
## dpkg-scanpackages error log.
> "${__TMP_PKG_DETAIL}"
> "${__LOG_DPKG_SCANPKG_MESSAGES}"
> "${__LOG_SCANNED_DIRS}"
> "${__TMP_PRELIM_DEB_LIST}"
> "${__TMP_SUFFICIENT_DEB_LIST}"
echo "Please wait. Scanning..."
echo
for (( i = 0 ; i < ${#deb_repos[@]} ; i++ )) ; do
echo "${deb_repos[i]}"
echo "$( command date +%F_%T ) :: START SCANNING DIRECTORY ${deb_repos[i]}" >> "${__LOG_DPKG_SCANPKG_MESSAGES}"
command dpkg-scanpackages "${deb_repos[i]}" /dev/null >> "${__TMP_PKG_DETAIL}" 2>> "${__LOG_DPKG_SCANPKG_MESSAGES}"
echo "${deb_repos[i]}" >> "${__LOG_SCANNED_DIRS}"
done
command egrep '^Filename:[[:space:]]+.*\.deb' "${__TMP_PKG_DETAIL}" \
| command sed -r 's,^Filename:[[:space:]]+,,;s,[[:space:]]+$,,' \
> "${__TMP_PRELIM_DEB_LIST}"
command awk -F'/' ' { __PKG_PATH[$NF]=$0 ; __PKG_PATH_COUNT[$NF]+=1 } \
END { \
for ( __W in __PKG_PATH ) { \
printf "%s\n" , __PKG_PATH[__W] ; \
if ( __PKG_PATH_COUNT[__W] > 1 ) \
printf "Package \"%s\" appears at %d locations, choosing the last.\n" , __W , __PKG_PATH_COUNT[__W] > "/dev/stderr" \
} \
}' "${__TMP_PRELIM_DEB_LIST}" > "${__TMP_SUFFICIENT_DEB_LIST}"
echo
echo "Scan complete."
echo
}
function _Prepare_Archivable_Disk_Repository(){
## Input: No input as args but requires the __TMP_SUFFICIENT_DEB_LIST file,
## __DIR_BASE and __LABEL_CODENAME directories.
## Output: Creates a directory tree and copies the .deb files listed in
## __TMP_SUFFICIENT_DEB_LIST file to the binary-i386/ directory
## in the root of archivable structure.
declare tmp_word=""
if [[ ! -e "${__TMP_SUFFICIENT_DEB_LIST}" ]] ; then
_Msg_Err "This function must be preceded by _Scan_Packages_Temp() function."
return 1
fi
if [[ -z "${__DIR_BASE}" ]] ; then
_Msg_Err "Use option -D to specify the path of the base directory to create the archive structure. See \`${__SCRIPT_NAME} -h'."
return 1
fi
if [[ -z "${__LABEL_CODENAME}" ]] ; then
_Msg_Err "Use option -l to specify the identification of the Ubuntu version you are using. See \`${__SCRIPT_NAME} -h'."
return 1
fi
echo
echo "Please make sure that \"${__DIR_BASE}\""
echo "is cleared of the stuffs you don't need. Any of the previously created"
echo "files/directories in there (except for a few) will not be removed or modified."
echo "Any excess of the packages, for new repositories added which are not in this"
echo "structure will be copied. The only files that will be modified are 'Packages'"
echo "files and its compressed forms (required by apt)."
read -n1 -p"Do you want to proceed (copying packages and generating 'Packages' file)? ( y/n ; default n ) : "
echo
if [[ "${REPLY}" != @(Y|y) ]] ; then
return 1
fi
if [[ ! -d "${__DIR_BASE}" ]] ; then
if ! command mkdir -p "${__DIR_BASE}" ; then
_Msg_Err "Could not create \"${__DIR_BASE}\" directory."
return 1
else
echo "Created \"${__DIR_BASE}\" directory."
fi
fi
for tmp_word in \
"${__DIR_BASE}/${__LABEL_CODENAME}" \
"${__DIR_BASE}/${__LABEL_CODENAME}/${__LABEL_BINARY_DIR}" \
"${__DIR_BASE}/${__LABEL_CODENAME}/${__LABEL_SOURCE_DIR}" ; do
if test ! -d "${tmp_word}" ; then
if command mkdir -p "${tmp_word}" ; then
echo "Created \"${tmp_word}\" directory."
else
_Msg_Err "Could not create \"${REPLY}\" directory."
return 1
fi
fi
done
## The README.info has to be placed in the root of the ISO image.
__DEFAULT_README="${__DIR_BASE}/README.info"
_Write_Default_Readme
echo
echo "Copying the essential .deb files to '${__LABEL_CODENAME}/${__LABEL_BINARY_DIR}/'"
## rsync --stats -h --progress --times --perms --group --owner --ignore-existing --verbose --files-from="${__TMP_SUFFICIENT_DEB_LIST}" "${__DIR_BASE}/${__LABEL_CODENAME}/${__LABEL_BINARY_DIR}/"
while read -r ; do
command cp --preserve=all --no-clobber --verbose -t "${__DIR_BASE}/${__LABEL_CODENAME}/${__LABEL_BINARY_DIR}/" "${REPLY}"
done < "${__TMP_SUFFICIENT_DEB_LIST}" && echo "Completed copying." || {
_Msg_Err "Encountered some error."
return 1
}
## Now begin creating the "Packages" file for `apt':
## THE SCAN WILL BE DONE FROM THE LEVEL OF ROOT FOR THE ISO IMAGE:
## cd "${__DIR_BASE}/${__LABEL_CODENAME}/" ## choosing instead to scan from the ISO root
cd "${__DIR_BASE}"
echo "$( command date +%F_%T ) :: START SCANNING ARCHIVING BINARY DIRECTORY ${deb_repos[i]}" >> "${__LOG_DPKG_SCANPKG_MESSAGES}"
echo
echo "Running \`dpkg-scanpackages' and generating 'Packages' file..."
# command dpkg-scanpackages "${__LABEL_BINARY_DIR}" /dev/null > "${__PACKAGE_DETAIL_FILE}"
command dpkg-scanpackages "${__LABEL_CODENAME}/${__LABEL_BINARY_DIR}" /dev/null > "${__PACKAGE_DETAIL_FILE}"
command gzip -c "${__PACKAGE_DETAIL_FILE}" > "${__PACKAGE_DETAIL_FILE}.gz"
command xz --suffix=.xz -kfz "${__PACKAGE_DETAIL_FILE}"
if [[ -e "${__PACKAGE_DETAIL_FILE}".gz && -e "${__PACKAGE_DETAIL_FILE}".xz ]] ; then
command rm "${__PACKAGE_DETAIL_FILE}"
fi
cd "${__SCRIPT_INITIAL_WDIR}"
}
if [[ ${#} -eq 0 ]] ; then
echo "See \`${__SCRIPT_NAME} -h'" >&2
exit 1
fi
declare __SCRIPT_OPTS=""
## If the option string starts with a colon (:), silent error reporting is used.
## The `getopts` builtin returns with value greater than zero on encountering
## end of option and assumes completion of option parsing which might not
## always be the case. The end of option is any invalid option or free operand.
## This can be explicitly marked with -- at the command line.
## Together with reading flag-type options and argument-type options with their
## arguments, getopts also shifts the positional parameters; the read options
## and arguments are popped out until end of options. The OPTIND which holds
## the index of next command line parameters, is initialized to 1 at each new
## invocation of the script, incremented for each call of `getopts`, but not
## reset and has to be reset manually (if required: re-processing of positional
## parameters). On encountering end of option, OPTIND is set to the position of
## the first non-option parameter, and __SCRIPT_OPTS is set to ?.
while getopts ":D:l:h" __SCRIPT_OPTS ; do
case "${__SCRIPT_OPTS}" in
D)
if [[ "${OPTARG}" == "-" ]] ; then
__DIR_BASE="${__PATH_DEFAULT_BASENAME%/}" ## also remove any trailing slash
else
__DIR_BASE="${OPTARG}" ## also remove any trailing slash
fi
;;
l)
__LABEL_CODENAME="${OPTARG}"
if [[ ! "${__LABEL_CODENAME}" =~ ^[^/]+$ ]] ; then
_Msg_Err "Error. The label specified with -l must not have a path separator character."
exit 1
fi
;;
h) _Show_Help ; exit 0 ;;
esac
## Silent error reporting: For "invalid option" errors, the __SCRIPT_OPTS
## is set to a question mark (?). For "argument to option not found"
## errors, the __SCRIPT_OPTS is set to a colon (:). In both error types,
## the option character is placed in OPTARG shell variable. [ This is
## not properly documented in bash man-page Ubuntu 16.04.4 LTS. ]
if [[ "${__SCRIPT_OPTS}" == "?" ]] ; then
echo "Option \"-${OPTARG}\" is not recognised."
echo "Try \`${0##*/} -h' for help." >&2
exit ${__ERR_GENERAL}
fi
if [[ "${__SCRIPT_OPTS}" == ":" ]] ; then
echo "Option -${OPTARG} requires an argument."
echo "Try \`${0##*/} -h' for help." >&2
exit ${__ERR_GENERAL}
fi
done
## Manual shifting of command line parameters might not be necessary with
## `getopts` as the builtin does that itself.
shift $(( ${OPTIND} - 1 ))
## Remember to reset OPTIND before processing next set of positional
## parameters.
# OPTIND=1
## Now, all that remains in $@ must be non-option arguments. (Remember that
## getopts will stop looking for more option on first non-option argument
## that it encounters.
## The first of the non-option argument must be the script ACTION specification
## and remaining all directories (repositories) of the Debian packages of
## applications and their dependencies.
## Getting the ACTION:
__SCRIPT_ACTION="${1}"
shift || {
_Msg_Err "Script action not specified. This must be the first non-option argument. See \`${__SCRIPT_NAME} -h'."
exit 1
}
## Assign the rest as repository directories:
__DIR_REPOSITORY=( "$@" )
## Reaching here, the contents of __SCRIPT_ACTION and __DIR_REPOSITORY are
## a must:
case "${__SCRIPT_ACTION}" in
prepare) ;;
list) ;;
*)
_Msg_Err "Error. The first non-option argument must specify an action. \"${__SCRIPT_ACTION}\" is not a valid action."
exit 1
;;
esac
if [[ ! -d "${__DIR_TMP}" ]] ; then
mkdir "${__DIR_TMP}"
fi
## Check the validity of __DIR_REPOSITORY
if [[ ${#__DIR_REPOSITORY[@]} -eq 0 ]] ; then
_Msg_Err "Error. A repository directory containing the Debian packages meant to archive must be specified."
exit 1
else
for (( __INDEX = 0 ; __INDEX < ${#__DIR_REPOSITORY[@]} ; __INDEX++ )) ; do
if [[ ! -d "${__DIR_REPOSITORY[${__INDEX}]}" ]] ; then
_Msg_Err "Directory \"${__DIR_REPOSITORY[${__INDEX}]}\" does not exist. (All options must preceed the non-option operands.)."
exit 1
fi
done
fi
## Codes to serve the commands:
case "${__SCRIPT_ACTION}" in
list)
if [[ -e "${__TMP_SUFFICIENT_DEB_LIST}" ]] ; then
echo "A list from a previous scan exists. The following directories were scanned:"
command cat "${__LOG_SCANNED_DIRS}"
echo
echo "Select n to see the previous list."
read -n1 -p"Perform a new scan ( y/n ; default n ): "
echo
if [[ "${REPLY}" == @(Y|y) ]] ; then
_Scan_Packages_Temp __DIR_REPOSITORY
echo "The following packages should be sufficient to archive:"
command cat "${__TMP_SUFFICIENT_DEB_LIST}"
echo
echo "Free space required: $( echo "scale=6; $( command cat "${__TMP_SUFFICIENT_DEB_LIST}" | command xargs -I{} du -b {} | command sed -r 's,^([[:digit:]]+)[[:space:]]+.*,\1,' | command paste -sd+ | command bc ) / 1024 ^ 3" | command bc ) GiB"
else
echo "The following packages should be sufficient to archive:"
command cat "${__TMP_SUFFICIENT_DEB_LIST}"
echo
echo "Free space required: $( echo "scale=6; $( command cat "${__TMP_SUFFICIENT_DEB_LIST}" | command xargs -I{} du -b {} | command sed -r 's,^([[:digit:]]+)[[:space:]]+.*,\1,' | command paste -sd+ | command bc ) / 1024 ^ 3" | command bc ) GiB"
fi
echo
fi
;;
prepare)
if [[ -z "${__DIR_BASE}" ]] ; then
_Msg_Err "Use option -D to specify the path of the base directory to create the archive structure. See \`${__SCRIPT_NAME} -h'."
exit 1
fi
if [[ -z "${__LABEL_CODENAME}" ]] ; then
_Msg_Err "Use option -l to specify the identification of the Ubuntu version you are using. See \`${__SCRIPT_NAME} -h'."
exit 1
fi
if [[ -e "${__TMP_SUFFICIENT_DEB_LIST}" ]] ; then
echo "A list from a previous scan exists. The following directories were scanned:"
command cat "${__LOG_SCANNED_DIRS}"
echo
echo "Select Y if you've adding more scan directories at the command line. (If"
echo "it's an entirely new list you SHOULD clear the contents of the"
echo "\"${__DIR_BASE}/${__LABEL_CODENAME}/${__LABEL_BINARY_DIR}/\" directory;"
echo "in that case select E, clear the directory and start again.)"
echo
read -n1 -p"Perform a new scan ( y/n/e/* ; default e ; E and any other exits ): "
echo
if [[ "${REPLY}" == @(Y|y) ]] ; then
_Scan_Packages_Temp __DIR_REPOSITORY
# echo "The following is the list of sufficient packages to archive:"
# command cat "${__TMP_SUFFICIENT_DEB_LIST}"
elif [[ "${REPLY}" == @(N|n) ]] ; then
true ## no-op
else
exit 0
fi
else
_Scan_Packages_Temp __DIR_REPOSITORY
fi
echo
echo "Total size of .deb packages to copy: $( echo "scale=6; $( command cat "${__TMP_SUFFICIENT_DEB_LIST}" | command xargs -I{} du -b {} | command sed -r 's,^([[:digit:]]+)[[:space:]]+.*,\1,' | command paste -sd+ | command bc ) / 1024 ^ 3" | command bc ) GiB."
echo "Make sure you have this amount of disk space available."
echo
_Prepare_Archivable_Disk_Repository
;;
esac
## End of script codes.
The steps to create an ISO file of the archive using xfburn
:
- Start the
xfburn
application. - Select Files -> New Data Composition
- Click Add button, browse to the BASE_DIRECTORY and select the repository directory, Packages.gz, Packages.xz, and README.info, and then at the bottom of the dialog, click on Add button.
- At the bottom of the window, the appropriate size and type of disk CD/DVD will be automatically selected. I've manually specified a 4.3GB DVD.
- Click on Proceed to Burn, the Burn Composition dialogue box will appear. In the options, check on Only create ISO, and then click on Burn Composition button.
Example - Creating a repository CD/DVD with boot-repair
and few other applications:
cd /tmp ; mkdir download_liveCD_essential_packages
cd download_liveCD_essential_packages
sudo add-apt-repository ppa:yannubuntu/boot-repair && sudo apt update
apt-get -y --print-uris install boot-repair clamav curl dos2unix extundelete fdupes foremost gnome-disk-utility gsmartcontrol iftop iotop konsole linkchecker links2 lynx nmap p7zip samba synaptic traceroute tree unrar vim whois | grep -o '\'http.*\' | tr "\'" " " >> packages_download.list
wget -i packages_download.list -c
wget -i packages_download.list -c ## confirm all downloads
bash ~/deb_cache_pkg_to_repository_image_v0.1.sh -D - -l lunar prepare /tmp/download_liveCD_essential_packages/ ## Assuming the script is in home directory
cd /tmp/prepare_apt_archive_for_ISO/
ls ## These are the files and directories you need to copy to the root of your .iso file. Next, use Xfburn or other apps to create the .iso file
Now, you can copy the required contents to the root of the ISO directory using GUI applications like xfburn
(recommended) or by using the following command:
genisoimage -allow-leading-dots -allow-lowercase -allow-multidot -iso-level 4 -max-iso9660-filenames -o /tmp/genisoimage__Ubuntu23.04_repo.iso -V "Ubuntu23.04_Repo" -R -uid 1000 -no-iso-translate /tmp/prepare_apt_archive_for_ISO/
or simply,
genisoimage -o /tmp/genisoimage__Ubuntu23.04_repo.iso -V "Ubuntu23.04_Repo" -R /tmp/prepare_apt_archive_for_ISO/
(Using Rock Ridge format in genisoimage
. These both commands seem to be working to create CD image, however for larger files, not accommodated in a CD image, you must also use tools like xfburn.)
Links:
Check out OpenRepo (https://github.com/openkilt/openrepo)
This open source project runs as a web server. You can upload packages to it and it will host it on your LAN for other servers to upgrade from.