Skip to content

Conversation

@eriktews
Copy link

@eriktews eriktews commented Mar 5, 2018

The current version of certbot-dns-rfc2136 requires the user to specify an DNS server using an IP address. This commit adds support for hostnmes too.

The current version of certbot-dns-rfc2136 requires the user to specify an DNS server using an IP address. This commit adds support for hostnmes too.
@noci2012
Copy link
Contributor

noci2012 commented Mar 9, 2018

gethostbyname will fail for IPv6, please use getaddrinfo see #5697

eriktews added 2 commits March 9, 2018 16:10
Switch from gethostbyname to getaddrinfo so that name servers that are
only reachable via IPv6 are also correctly handled.
@eriktews
Copy link
Author

eriktews commented Mar 9, 2018

I have updated my branch to take this into account and also improved the behaviour regarding exception handling should the lookup fail.

@noci2012
Copy link
Contributor

I closed the other one this is now a better change.

raise errors.PluginError('Received response from server: {0}'
.format(dns.rcode.to_text(rcode)))

def _get_ip_for_hostname(self, hostname):
Copy link
Collaborator

@zjs zjs Mar 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a way to write some meaningful unit tests for this method? (Perhaps one where hostname is an IP address, one where hostname is expected to resolve successfully, and one where hostname is expected to fail to resolve?)

Among other benefits, it would be nice for Travis to continually verify that the socket.getaddrinfo invocation below works as expected on all supported platforms and Python versions.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think there is at least some minimal testing we can do here. I have written some basic tests that do not depend on external services:

eriktews@c26cee8

Also the spelling has been corrected.

My branch has bee updated with those improvements.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately Travis CI does not yet support IPv6, so the IPv6 test has to be disabled for now. It passes fine on my local system.


def _get_ip_for_hostname(self, hostname):
"""
Get an IP address for a supplied hostnamed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/hostnamed/hostname/

try:
self.server = self._get_ip_for_hostname(server)
except Exception as e:
raise errors.PluginError("Failed to resolv {0}: {1}".format(server, e))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/resolv/resolve/

Those unit tests will check whether name servers can be correctly resolved by certbot-dns-rfc2136 and whether requests for invalid server names or invalid IP addresses are correctly reported as errors.
Copy link
Collaborator

@zjs zjs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding some more tests!

def test_ipv4(self):
self.client("127.0.0.1")

# Travis Ci does not support IPv6 yet, so this test is disabled
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It has been a while since I've written Python unit tests, so this could be a crazy suggestion, but...

Would it be possible to use something like @unittest.skip("Travis CI does not support IPv6 yet, so this test is disabled") instead of just commenting it out? Hopefully this would cause the fact that the unit test is skipped to be reported in the output, to help remind anyone making changes in this area that the test is being skipped.

Even better might be to use something like @unittest.skipIf(???, "This system does not support IPv6, so this test will be skipped"), assuming there's a simple conditional check for IPv6 support you can substitute for the ???. If that works like I'm imagining, it would let people easily test IPv6 locally (on systems that support it) and automatically start testing it in Travis CI if support is ever added there.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So determining whether IPv6 is supported or not in python is possible, but rather complex (and there are various ways how IPv6 might not be available, such as it is supported by the kernel but maybe just not configured on the loopback interface) and I would like to avoid this for now.

However there is an alternative solution:

    @unittest.skipIf("TRAVIS" in os.environ,
                     "IPv6 is not always supported in Travis, IPv6 test skipped")

This will disable the IPv6 check on Travis for now. As soon as the overall Travis configuration/environment will support IPv6, we can remove this and enable the test again.

Also, I improved the general tests a bit by comparing the server property with the expected values.

Now the sucess of the DNS lookup is checked by comparing the server property with the expected value. Also the IPv6 test is only skipped when the tests are running in Travis since Travis does not always support IPv6.
@eriktews
Copy link
Author

May I support you any further with that?

# exception would have been raised by getaddrinfo.

# Just return the first address in string representation.
return results[0][4][0]
Copy link

@laendle laendle Mar 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not reliable, that is results[0][4][0] is invalid when len(results[0][4]) > 1, see below...

Test case (Windows 10),

  1. Install VirtualBox. (It will add multiple adaptors).
  2. In a console window run, ipconfig -all and observe a number of NICs with IPv4 and IPv6
  3. The proposed function _get_ip_for_hostname(server)..
    a) will not reliably return an address for the NIC that is currently used for outbound network comms.
    b) will only return an IPv6 address when both IPv4 and IPv6 exist
    c) socket.gethostbyname(server) != _get_ip_for_hostname(server)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the current man-page says:

      There are several reasons why the linked list may have  more  than  one
       addrinfo structure, including: the network host is multihomed, accessi‐
       ble over multiple protocols (e.g., both AF_INET and AF_INET6);  or  the
       same  service  is available from multiple socket types (one SOCK_STREAM
       address and another SOCK_DGRAM address, for  example).   Normally,  the
       application  should  try using the addresses in the order in which they
       are returned.   The  sorting  function  used  within  getaddrinfo()  is
       defined  in  RFC 3484; the order can be tweaked for a particular system
       by editing /etc/gai.conf (available since glibc 2.5).

So the first element should be the best choice when you pick only a single entry.

Since it looks like we cannot get the best address for the server from the list without applying a lot of additional magic, I would recommend I change the code to return all entries. The main code will then try all entries until the DNS request succeeded for one of them. As a benefit, we can also change the configuration file in a way that you can specify multiple IP addresses there and have the additional ones as a fallback.

What do you think of that?

Copy link

@laendle laendle Mar 31, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should try the test case for yourself as speculation may hinder your success. For example, each item ipconfig -all returns show a valid DNS too, even if virtual. You would observe this yourself if you tested, and make a more informed design decision as to whether a test by DNS is viable (given multi DNS can exist - consider too that VMWare workstation installs a virtual DNS by default for its vNICs) ;)

Copy link
Contributor

@noci2012 noci2012 Jun 4, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should not return an address for outgoing connections but the IP address of the Nameserver where an update is needed. That should come from DNS, or whatever the resolver used to translate the hostname to a number. To Help search, one supplies the intended portnumber (ic. 53).
Also if the hostname contains an IPv4 address gethostbyname() does work, if hostname contains an IPv6 address gethostbyname() just fails.
That is the core problem is: if socket.gethostbyname() is passed an IPv6 address it does NOT copy the address to the h_addr field. like it does for IPv4 addresses, it just fails.
Therefor gethostbyname() is invalidated by reality. (in places where there is no ipv4 address available as a "backup")

So if getaddrinfo() (used by _gethostbyname(server)) cannot be used.., it would be better to drop any name lookup and just validate if it is either a valid IPv4 address or IPv6 address and create the structures themselves.
And why would socket.gethostbyname(server) return then same info as getaddrinfo(...server...) if they allow different sorting orders and getaddrinfo().. also supplies portinfo...

@bmw
Copy link
Member

bmw commented Sep 12, 2018

@sydneyli, said she could take a look at this.

@kpfleming
Copy link

If there's any way I can help move this along, please let me know. I just ran into this same problem today and would very much prefer to use a hostname instead of a hardcoded address.

@kpfleming
Copy link

@eriktews can you update this branch to resolve the merge conflicts and let the checks run again?

@eriktews
Copy link
Author

Hi, it's done. With some luck, I can later try to improve the patch so that it will try multiple DNS servers as well.

@noci2012
Copy link
Contributor

@eriktews why should it update multiple servers?
A slave is updated though the master DNS. So imho there is only one server to update..., that one will update all others. [ if setup correctly within a few seconds ].

@eriktews
Copy link
Author

@eriktews why should it update multiple servers?
A slave is updated though the master DNS. So imho there is only one server to update..., that one will update all others. [ if setup correctly within a few seconds ].

Hi, it might happen that the name of a DNS server resolves to more than one IP address, for example one IPv6 and one IPv4 address. Maybe the system doesn't have a good IPv6 connection, so contacting the server vie IPv6 fails. When that happens, it would be a good alternative to try the IPv4 address as well.

Also when there are multiple IPv4 addresses (a server cluster) and one of the servers is down, we might continue trying other addresses as well till the update succeeds with one of the servers.

@noci2012
Copy link
Contributor

IMHO:
If IPv6 connectivity is not good then either the local system should not request AAAA or the DNS server should not publish an AAAA. If it fails due to IPv6 connectivity issues there are moe problems then only being unable to update DNS.., also why would IPv4 work while IPv6 fails if properly setup - so not convincing.
In the case of cluster addresses if an address is published it "SHOULD" work on the published address. If it doesn't there are bigger issues with the cluster, where parts might fail.
So if more than one address is published on a DNS server, then they are all equal and one update should suffice.

@m0namon
Copy link
Contributor

m0namon commented May 28, 2020

Is there any reason not to use the stub resolver in dnspython ? It's already a dependency, and I imagine it might be more reliable than rolling our own.

@brianjmurrell
Copy link

Any updates here?

@mmonaco
Copy link

mmonaco commented Nov 30, 2021

Is there some more history here? Using a server hostname for rfc2136 worked just fine for me until recently. Is this due to an underlying library change? I found this from #7871 but am surprised that this dates back to 2018.

@brianjmurrell
Copy link

Can any progress be made here? What is blocking this PR from landing?

@brianjmurrell
Copy link

Why has this PR been open for over 5 years and not landed? Nothing has been done with it since I last asked about it a year ago.

@gjongenelen
Copy link

Facing the same prolonged inconvenience, and I see a 5-year-old pull request... Any chance to speed things up?

@certbot certbot deleted a comment from fda77 Dec 5, 2024
@bmw bmw changed the base branch from master to main February 10, 2025 18:14
@bmw bmw unassigned m0namon Feb 11, 2025
@bmw bmw self-assigned this Sep 24, 2025
@bmw bmw self-requested a review September 24, 2025 23:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.