Q：Why does the bindresvport_sa function use ports 600 ~ 1023?

[Haroldll]

When ganesha tries to connect to other services, such as rpcbind, the ports 600~1023 are used when calling the bindresvport_sa interface(assuming that AF_LOCAL is not used), is there any particular reason for this, can't we just have the system assign a random source port?

code see:

#ifdef __linux__
#define STARTPORT 600
#define LOWPORT 512
#define ENDPORT (IPPORT_RESERVED - 1)
#define NPORTS  (ENDPORT - STARTPORT + 1)
int
bindresvport_sa(int sd, struct sockaddr *sa)
{
............
	u_int16_t *portp;
	static u_int16_t port;
	static short startport = STARTPORT;
	socklen_t salen;
	int nports = ENDPORT - startport + 1;
	int endport = ENDPORT;
	int i;
............

	if (port == 0)
		port = (getpid() % NPORTS) + STARTPORT;

	res = -1;
	errno = EADDRINUSE;
 again:
	for (i = 0; i < nports; ++i) {
		*portp = htons(port++);
		if (port > endport)
			port = startport;
		res = bind(sd, sa, salen);
		if (res >= 0 || errno != EADDRINUSE)
			break;
	}
	if (i == nports && startport != LOWPORT) {
		startport = LOWPORT;
		endport = STARTPORT - 1;
		nports = STARTPORT - LOWPORT;
		port = LOWPORT + port % (STARTPORT - LOWPORT);
		goto again;
	}
	return (res);
}

Ganesha uses a lot of ports in this segment when it starts up, binds one for every request, and then releases it, but there is a time_wait time, and the next time it picks a new port, all of which occupy a lot of ports. This time may cause the bind port to fail for application services that use this port.

So can just set port to 0 and let the system assign a port, like this:

    sin->sin_port = 0;
    sa->sa_family = AF_INET;
    res = bind(sd, sa, salen);

Or is there some kind of restriction that the ports here must be 600~1023?

[ffilz]

See https://man7.org/linux/man-pages/man3/bindresvport.3.html

The range is so that some of the reserved ports (under 1024) are reserved for assigned services.

I'm not quite sure where Ganesha is calling bindresvport though, what is the stack back trace?

[Haroldll]

I'm not quite sure where Ganesha is calling bindresvport though, what is the stack back trace?

Thanks for the reply.
My scenario is unique, rpcbind is running on the host while Ganesha is running in a container utilizing the host network.
Since ganesha and rpcbind are not in the same OS, can not use AF_LOCAL, registering the rpc service when ganesha starts will result in the following stack call:
nfs_Init_svc
Register_program
__Register_program
svc_reg
rpcb_set
local_rpcb
getclnthandle
clnt_tli_ncreate
bindresvport

[Haroldll]

When the ganesha of this container is restarted frequently for a short period of time, this interface is called to bind a large number of ports because it needs to logout rpc and register rpc each time it is started, and it also needs to logout rpc when it is stopped. Sometimes the port 875 is occupied, although it is immediately released, but there is a period of time_wait state, the next time the startup will fail to bind port 875, the quota port default 875.

The rpc service registration, each time, is registering one version of one service, one network protocol, so registering all the services, needs to be executed many times. And each time it will apply for a new port occupation, and then release, this mechanism will occupy a large number of ports for a short period of time, of course, soon after all will be released.

Of course, it's just my weird way of using it that causes this problem.

/**

• @brief Default RQUOTA port.
  */
  #define RQUOTA_PORT 875
  875, can we change it to another port?

[ffilz]

Port 875 for rquotad is specified in /etc/services. You could configure it differently. Or, if you don't actually need RQUOTA you could disable it.

Why are you restarting Ganesha so often?

I suppose we should also look at making it so Ganesha does all the rpcbind registrations with one connection...

[Haroldll]

Why are you restarting Ganesha so often?

I did some reliability testing and repeated reboots just to test.

I suppose we should also look at making it so Ganesha does all the rpcbind registrations with one connection...

Yes, that sounds great. It's more efficient and less resource intensive to use a single connection.

[ffilz]

Open a Ganesha issue for this, and ideally consider submitting a patch.

[ffilz]

Another option here that I have proposed in the past if for Ganesha to take over the rpcbind service and not use RPC to communicate with it... This idea is somewhat less exciting than Ganesha taking over rpc.statd...

[ffilz]

Otherwise, can we close this? If desired, please open a Ganesha issue