Archive
FreeBSD Howto Mix Match Packages and Ports
This document is not for best practices and/or proof of concept
This is a solution for my simple problem: I’m managing FreeBSD server with packages not ports but default package configuration doesn’t meet my needs.
Note: For mariadb there is already a DBD package: p5-DBD-MariaDB but it still requires mysql57-client
Current Scenario :
I use mariadb instead of mysql but FreeBSD 12.1-RELEASE defaults to mysql 5.7 meaning packages are build based on this version of mysql so:
I’ve mariadb 10.4 installed and i need p5-DBD-mysql package for some perl script to run
if i try to install p5-DBD-mysql from packages:
root@bsd12:~ # pkg install p5-DBD-mysql
Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. The following 10 package(s) will be affected (of 0 checked): New packages to be INSTALLED: groff: 1.22.4_3 libevent: 2.1.12 libpaper: 1.1.24.4 mysql57-client: 5.7.31_1 p5-DBD-mysql: 4.050 p5-DBI: 1.643 protobuf: 3.13.0,1 psutils: 1.17_5 uchardet: 0.0.7 Installed packages to be UPGRADED: perl5: 5.30.3 -> 5.32.0 Number of packages to be installed: 9 Number of packages to be upgraded: 1 The process will require 100 MiB more space. 23 MiB to be downloaded. Proceed with this action? [y/N]: y [1/10] Fetching p5-DBD-mysql-4.050.txz: 100% 99 KiB 101.3kB/s 00:01 [2/10] Fetching perl5-5.32.0.txz: 100% 14 MiB 14.9MB/s 00:01 [3/10] Fetching libpaper-1.1.24.4.txz: 100% 24 KiB 24.5kB/s 00:01 [4/10] Fetching p5-DBI-1.643.txz: 100% 705 KiB 721.6kB/s 00:01 [5/10] Fetching mysql57-client-5.7.31_1.txz: 100% 2 MiB 1.9MB/s 00:01 [6/10] Fetching groff-1.22.4_3.txz: 100% 3 MiB 2.9MB/s 00:01 [7/10] Fetching uchardet-0.0.7.txz: 100% 108 KiB 111.0kB/s 00:01 [8/10] Fetching psutils-1.17_5.txz: 100% 57 KiB 58.0kB/s 00:01 [9/10] Fetching protobuf-3.13.0,1.txz: 100% 3 MiB 2.9MB/s 00:01 [10/10] Fetching libevent-2.1.12.txz: 100% 320 KiB 327.6kB/s 00:01 Checking integrity... done (1 conflicting) - mysql57-client-5.7.31_1 conflicts with mariadb104-client-10.4.13_4 on /usr/local/bin/mysql
As you see above it conflicts with my mariadb and continues as below:
Checking integrity... done (0 conflicting) Conflicts with the existing packages have been found. One more solver iteration is needed to resolve them. The following 12 package(s) will be affected (of 0 checked): New packages to be INSTALLED: groff: 1.22.4_3 libevent: 2.1.12 libpaper: 1.1.24.4 mysql57-client: 5.7.31_1 p5-DBD-mysql: 4.050 p5-DBI: 1.643 protobuf: 3.13.0,1 psutils: 1.17_5 uchardet: 0.0.7 Installed packages to be UPGRADED: perl5: 5.30.3 -> 5.32.0 Number of packages to be installed: 9 Number of packages to be upgraded: 1 The process will require 100 MiB more space. Proceed with this action? [y/N]:
Now if i click on “Yes” here pkg will remove my mariadb and install mysql57-client so of course i click on “No”
Solution is change FreeBSD defaults and make it depend on Mariadb104 instead of mysql. For this we need to use ports
About Ports:
There are a lot of flame around mixing pkgs and ports i wont go into that. But in my case its mandatory and pretty harmless because:
I dont use HEAD ports but use Quarterly which is same as pkg version.
Little bit background; pkg uses quarterly updates repo as default on the other hand port users tend to use HEAD branch meaning kind of rolling release.
Port management tools:
Portsnap: uses HEAD branch as default, useless for me
Poudriere: Uses jails, a lot of resources and complicated for single pkg. Helpful if you are going to create batch of packages, have tens or hundreds of FreeBSD servers and want to create your own central repo server.
Synth: Simplified package repository builder, doesn’t require that much resource uses base system as building platform and creates dynamic jails during each build.
I will use svnlite, simple enough for my needs. Basicaly:
1) Change freebsd make defaults to use mariadb instead of mysql
2) Download ports tree
3) create p5-DBD-mysql package from ports
4) install p5-DBD-mysql pkg
5) test
root@bsd12:~ # echo "DEFAULT_VERSIONS= mysql=10.4m" >> /etc/make.conf
Download ports from quarterly repo :
// if you already created ports with another port management tool remove ports directory
root@bsd12:~ # svnlite checkout https://svn.freebsd.org/ports/branches/2020Q4 /usr/ports
it will take a while to download ports ( uses ~1.2GB ) once its done
root@bsd12:~ # cd /usr/ports/
root@bsd12:~ # make fetchindex
root@bsd12:~ # make search name=”p5-DBD-mysql”
Above command is to search for a port in ports tree
root@bsd12:~ # cd databases/p5-DBD-mysql
root@bsd12:~ # make all-depends-list | sort
Now above command shows all dependencies and if you scroll up, you will see now mariadb is listed as dependency instead of mysql
root@bsd12:~ # make package
root@bsd12:~ # pkg add work/pkg/p5-DBD-mysql-4.050.txz
And you are done. You can test installation with below perl script that i copied from cpan site.
Test Script:
#!/usr/local/bin/perl
use strict;
use warnings;
use DBI;
# Connect to the database.
my $dbh = DBI->connect("DBI:mysql:database=test;host=localhost",
"user", "password",
{'RaiseError' => 1});
# Drop table 'foo'. This may fail, if 'foo' doesn't exist
# Thus we put an eval around it.
eval { $dbh->do("DROP TABLE foo") };
print "Dropping foo failed: $@\n" if $@;
# Create a new table 'foo'. This must not fail, thus we don't
# catch errors.
$dbh->do("CREATE TABLE foo (id INTEGER, name VARCHAR(20))");
# INSERT some data into 'foo'. We are using $dbh->quote() for
# quoting the name.
$dbh->do("INSERT INTO foo VALUES (1, " . $dbh->quote("Tim") . ")");
# same thing, but using placeholders (recommended!)
$dbh->do("INSERT INTO foo VALUES (?, ?)", undef, 2, "Jochen");
# now retrieve data from the table.
my $sth = $dbh->prepare("SELECT * FROM foo");
$sth->execute();
while (my $ref = $sth->fetchrow_hashref()) {
print "Found a row: id = $ref->{'id'}, name = $ref->{'name'}\n";
}
$sth->finish();
# Disconnect from the database.
$dbh->disconnect();
And last step is to lock the pkg so it will not be affected during update process
root@bsd12:~ # pkg lock p5-DBD-mysql p5-DBD-mysql-4.050: lock this package? [y/N]: y Locking p5-DBD-mysql-4.050
Freebsd 12.1-RELEASE Mariadb installation
root@freebitch12:/etc # pkg search mariadb
mariadb-connector-c-3.1.9 MariaDB database connector for C mariadb-connector-odbc-3.1.7_1 MariaDB database connector for odbc mariadb103-client-10.3.23 Multithreaded SQL database (client) mariadb103-server-10.3.23 Multithreaded SQL database (server) mariadb104-client-10.4.13_4 Multithreaded SQL database (client) mariadb104-server-10.4.13_4 Multithreaded SQL database (server) rubygem-azure_mgmt_mariadb-0.17.3 Microsoft Azure Microsoft Azure MariaDB Library for Ruby Client Library for Ruby
root@freebitch12:/etc # pkg install mariadb104-server mariadb104-client
Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. The following 12 package(s) will be affected (of 0 checked): New packages to be INSTALLED: bash: 5.0.17 boost-libs: 1.72.0_2 galera26: 26.4.5 icu: 67.1,1 libedit: 3.1.20191231,1 liblz4: 1.9.2_1,1 mariadb104-client: 10.4.13_4 mariadb104-server: 10.4.13_4 rsync: 3.2.3 unixODBC: 2.3.7 xxhash: 0.7.4 zstd: 1.4.5 Number of packages to be installed: 12 The process will require 483 MiB more space. 60 MiB to be downloaded. Proceed with this action? [y/N]: y .....
root@freebitch12:/etc # vim /etc/rc.conf
mysql_enable="yes" mysql_pidfile="/var/db/mysql/mysql.pid" mysql_optfile="/usr/local/etc/mysql/my.cnf"
root@freebitch12:/etc # service mysql-server start
Installing MariaDB/MySQL system tables in '/var/db/mysql' ... OK To start mysqld at boot time you have to copy support-files/mysql.server to the right place for your system Two all-privilege accounts were created. One is root@localhost, it has no password, but you need to be system 'root' user to connect. Use, for example, sudo mysql The second is mysql@localhost, it has no password either, but you need to be the system 'mysql' user to connect. After connecting you can set the password, if you would need to be able to connect as any of these users with a password and without sudo See the MariaDB Knowledgebase at http://mariadb.com/kb or the MySQL manual for more instructions. You can start the MariaDB daemon with: cd '/usr/local' ; /usr/local/bin/mysqld_safe --datadir='/var/db/mysql' You can test the MariaDB daemon with mysql-test-run.pl cd '/usr/local/mysql-test' ; perl mysql-test-run.pl Please report any problems at http://mariadb.org/jira The latest information about MariaDB is available at http://mariadb.org/. You can find additional information about the MySQL part at: http://dev.mysql.com Consider joining MariaDB's strong and vibrant community:Get InvolvedStarting mysql.
root@freebitch12:/etc # service mysql-server status
mysql is not running.
root@freebitch12:/etc # tail /var/log/mysql/mysqld.err
2020-10-02 19:02:13 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB. 2020-10-02 19:02:13 0 [Note] InnoDB: Waiting for purge to start 2020-10-02 19:02:13 0 [Note] InnoDB: 10.4.13 started; log sequence number 60972; transaction id 21 2020-10-02 19:02:13 0 [Note] InnoDB: Loading buffer pool(s) from /var/db/mysql/ib_buffer_pool 2020-10-02 19:02:13 0 [Note] InnoDB: Buffer pool(s) load completed at 201002 19:02:13 2020-10-02 19:02:13 0 [Note] Plugin 'FEEDBACK' is disabled. 2020-10-02 19:02:13 0 [Note] Server socket created on IP: '127.0.0.1'. 2020-10-02 19:02:13 0 [ERROR] Can't start server : Bind on unix socket: Permission denied 2020-10-02 19:02:13 0 [ERROR] Do you already have another mysqld server running on socket: /var/run/mysql/mysql.sock ? 2020-10-02 19:02:13 0 [ERROR] Aborting
root@freebitch12:/etc # ls -alh /var/run/mysql/
total 8 drwxr-xr-x 2 root wheel 512B Oct 2 19:19 . drwxr-xr-x 7 root wheel 512B Oct 2 19:00 ..
root@freebitch12:/etc # chown mysql:mysql /var/run/mysql/
root@freebitch12:/etc # service mysql-server start
Starting mysql.
root@freebitch12:/etc # service mysql-server status
mysql is running as pid 91966.
root@freebitch12:~ # mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MariaDB to secure it, we'll need the current
password for the root user. If you've just installed MariaDB, and
haven't set the root password yet, you should just press enter here.
Enter current password for root (enter for none):
OK, successfully used password, moving on...
Setting the root password or using the unix_socket ensures that nobody
can log into the MariaDB root user without the proper authorisation.
You already have your root account protected, so you can safely answer 'n'.
Switch to unix_socket authentication [Y/n] n
... skipping.
You already have your root account protected, so you can safely answer 'n'.
Change the root password? [Y/n] y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
... Success!
By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them. This is intended only for testing, and to make the installation
go a bit smoother. You should remove them before moving into a
production environment.
Remove anonymous users? [Y/n] y
... Success!
Normally, root should only be allowed to connect from 'localhost'. This
ensures that someone cannot guess at the root password from the network.
Disallow root login remotely? [Y/n] y
... Success!
By default, MariaDB comes with a database named 'test' that anyone can
access. This is also intended only for testing, and should be removed
before moving into a production environment.
Remove test database and access to it? [Y/n] y
- Dropping test database...
... Success!
- Removing privileges on test database...
... Success!
Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.
Reload privilege tables now? [Y/n] y
... Success!
Cleaning up...
All done! If you've completed all of the above steps, your MariaDB
installation should now be secure.
Thanks for using MariaDB!
Possible Problem
** if you receive below error when you try to run mysql_secure_installation script just restart your server and re-run mysql_secure_installation , you’ll be fine
root@freebitch12:/etc # mysql_secure_installation
NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
SERVERS IN PRODUCTION USE! PLEASE READ EACH STEP CAREFULLY!
In order to log into MariaDB to secure it, we'll need the current
password for the root user. If you've just installed MariaDB, and
haven't set the root password yet, you should just press enter here.
Enter current password for root (enter for none):
error: Config file /etc/mail/.my.cnf.92133 in invalid location, please move to or merge with /usr/local/etc/mail/.my.cnf.92133
Fatal error in defaults handling. Program aborted
Enter current password for root (enter for none):
Aborting!
Cleaning up...
Freebsd 12.1-RELEASE Sendmail SMTP-Auth Configuration
// Total time 5-10 minutes
// Important: Remove any modifications you make to /etc/mail/freebsd.mc
NOTE: This documents explains how to compile default sendmail to support smtp-auth however; you also have the option to install sendmail pkg from packages collection. This way you don't have to compile sendmail, sendmail version in pkg collection is already compiled with smtp auth enabled!!! When you install sendmail from pkgs collection, system will also install necessary additional pkgs (cyrus-sasl & cyrus-sasl-saslauthd) All you have to do is;
root@freebitch12:/ # pkg install sendmail root@freebitch12:/ # cd /usr/local/etc/mail && cp mailer.conf.sendmail mailer.conf You still need to follow below configurations in order to enable smtp-auth for sendmail; Except you do NOT need to compile sendmail and install cyrus-sasl cyrus-sasl-saslauthd
// check sendmail compiled options
root@freebitch12:/etc/mail # sendmail -d0.1 -bv root
Version 8.15.2 Compiled with: DNSMAP IPV6_FULL LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8 MIME8TO7 NAMED_BIND NETINET NETINET6 NETUNIX NEWDB NIS PIPELINING SCANF STARTTLS TCPWRAPPERS USERDB XDEBUG Warning: Option: AuthMechanisms requires SASL support (-DSASL) ============ SYSTEM IDENTITY (after readcf) ============ (short domain name) $w = freebitch12 (canonical domain name) $j = freebitch12.localrouter (subdomain name) $m = localrouter (node name) $k = freebitch12 ======================================================== root... deliverable: mailer local, user root
// No SASLv2 support
root@freebitch12:/etc # pkg install cyrus-sasl cyrus-sasl-saslauthd
Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. Checking integrity... done (0 conflicting) The following 2 package(s) will be affected (of 0 checked): New packages to be INSTALLED: cyrus-sasl: 2.1.27_1 cyrus-sasl-saslauthd: 2.1.27_1 Number of packages to be installed: 2 The process will require 8 MiB more space. Proceed with this action? [y/N]: y [1/2] Installing cyrus-sasl-2.1.27_1... *** Updated user `cyrus'. [1/2] Extracting cyrus-sasl-2.1.27_1: 100% [2/2] Installing cyrus-sasl-saslauthd-2.1.27_1... [2/2] Extracting cyrus-sasl-saslauthd-2.1.27_1: 100% .....
// Enable sendmail and saslauthd
root@freebitch12:/etc # vim /etc/rc.conf
sendmail_enable="YES" saslauthd_enable="YES"
// copy freebsd source tree to /usr/src
// required to compile sendmail
root@freebitch12:/etc # svnlite checkout https://svn.freebsd.org/base/releng/12.1/ /usr/src
.... A /usr/src/tools/bus_space/Python/Makefile A /usr/src/tools/bus_space/C/libbus.h A /usr/src/tools/bus_space/bus.c A /usr/src/tools/sched A /usr/src/tools/ifnet A /usr/src/tools/LibraryReport .... A /usr/src/README A /usr/src/Makefile.sys.inc U /usr/src Checked out revision 366374.
// create sendmail make.conf file
root@freebitch12:/etc # vim /etc/make.conf
SENDMAIL_CFLAGS=-I/usr/local/include/sasl -DSASL SENDMAIL_LDADD=/usr/local/lib/libsasl2.so
//compile sendmail with auth support, takes 2-3 minutes
root@freebitch12:/etc # cd /usr/src/lib/libsmutil
root@freebitch12:/usr/src/lib/libsmutil # make cleandir && make obj && make
root@freebitch12:/usr/src/lib/libsmutil # cd /usr/src/lib/libsm
root@freebitch12:/usr/src/lib/libsm # make cleandir && make obj && make
root@freebitch12:/usr/src/lib/libsm # cd /usr/src/usr.sbin/sendmail
root@freebitch12:/usr/src/usr.sbin/sendmail # make cleandir && make obj && make && make install
root@freebitch12:/usr/src/usr.sbin/sendmail # cd /etc/mail
root@freebitch12:/etc/mail # make
// this will create copy of freebsd.mc, freebsd.submit.cf … as hostname.mc, hostname.submit.cf …
// dont make changes to freebsd.mc, make changes to hostname.mc
// in my case freebitch12.mc ……
root@freebitch12:/etc/mail # vim freebitch12.mc
define(`confSMTP_LOGIN_MSG', `freebitch12')dnl define(`confAUTH_OPTIONS', `A p y')dnl TRUST_AUTH_MECH(`LOGIN PLAIN')dnl define(`confAUTH_MECHANISMS', `LOGIN PLAIN')dnl
// add any other option you require to hostname.mc
// no need to add DAEMON_OPTIONS(`Port=smtp,Addr=0.0.0.0, Name=MTA’)dnl
// When you enable sendmail in rc.conf it starts listening all interfaces
// Actually if you add this option sendmail wont be able to bind address and would not start
// define(`confAUTH_OPTIONS’, `A p y’)dnl
// this option forces LOGIN/PLAIN SMTP-AUTH after encryption has been established in a TLS tunnel.
// this means you will not see 250-AUTH LOGIN PLAIN if you telnet to port 25
// A is a workaround for broken MTAs that do not implement RFC 2554.
// p option tells sendmail: “don’t permit mechanisms susceptible to simple passive attack (e.g., LOGIN, PLAIN), unless a security layer (think TLS tunnel) is active.”
// you need to use # openssl s_client -connect localhost:25 -starttls smtp to see:
// 250-AUTH LOGIN PLAIN
// y option prohibits anonymous logins
root@freebitch12:/etc/mail # make install
// for each change to mc file you need to run make install like old linux or you can add make install to sendmail start script
// check if SASLv2 support is added
root@freebitch12:/etc/mail # sendmail -d0.1 -bv root
Version 8.15.2 Compiled with: DNSMAP IPV6_FULL LOG MAP_REGEX MATCHGECOS MILTER MIME7TO8 MIME8TO7 NAMED_BIND NETINET NETINET6 NETUNIX NEWDB NIS PIPELINING SASLv2 SCANF STARTTLS TCPWRAPPERS TLS_EC TLS_VRFY_PER_CTX USERDB XDEBUG ============ SYSTEM IDENTITY (after readcf) ============ (short domain name) $w = freebitch12 (canonical domain name) $j = freebitch12.localrouter (subdomain name) $m = localrouter (node name) $k = freebitch12 ======================================================== root... deliverable: mailer local, user root
root@freebitch12:/etc/mail # service sendmail restart
// test AUTH LOGIN PLAIN support with plain telnet, returns no support
root@freebbitch12:~ # telnet localhost 25
Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. 220 freebbitch12 ESMTP ehlo test 250-freebbitch12.localrouter Hello localhost [127.0.0.1], pleased to meet you 250-ENHANCEDSTATUSCODES 250-PIPELINING 250-8BITMIME 250-SIZE 250-DSN 250-ETRN 250-STARTTLS 250-DELIVERBY 250 HELP quit
// test AUTH LOGIN PLAIN support with openssl, returns support
root@freebbitch12:~ # openssl s_client -connect localhost:25 -starttls smtp
....
lots of certificate info
....
Extended master secret: no
Max Early Data: 0
---
read R BLOCK
ehlo test
250-freebbitch12.localrouter Hello localhost [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE
250-DSN
250-ETRN
250-AUTH LOGIN PLAIN
250-DELIVERBY
250 HELP
quit
221 2.0.0 freebbitch12.localrouter closing connection
Notes From Kuku 