-
-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Also allow hostnames instead of just IP addresses #5664
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
The current version of certbot-dns-rfc2136 requires the user to specify an DNS server using an IP address. This commit adds support for hostnmes too.
|
gethostbyname will fail for IPv6, please use getaddrinfo see #5697 |
Switch from gethostbyname to getaddrinfo so that name servers that are only reachable via IPv6 are also correctly handled.
|
I have updated my branch to take this into account and also improved the behaviour regarding exception handling should the lookup fail. |
|
I closed the other one this is now a better change. |
| raise errors.PluginError('Received response from server: {0}' | ||
| .format(dns.rcode.to_text(rcode))) | ||
|
|
||
| def _get_ip_for_hostname(self, hostname): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way to write some meaningful unit tests for this method? (Perhaps one where hostname is an IP address, one where hostname is expected to resolve successfully, and one where hostname is expected to fail to resolve?)
Among other benefits, it would be nice for Travis to continually verify that the socket.getaddrinfo invocation below works as expected on all supported platforms and Python versions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I think there is at least some minimal testing we can do here. I have written some basic tests that do not depend on external services:
Also the spelling has been corrected.
My branch has bee updated with those improvements.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately Travis CI does not yet support IPv6, so the IPv6 test has to be disabled for now. It passes fine on my local system.
|
|
||
| def _get_ip_for_hostname(self, hostname): | ||
| """ | ||
| Get an IP address for a supplied hostnamed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: s/hostnamed/hostname/
| try: | ||
| self.server = self._get_ip_for_hostname(server) | ||
| except Exception as e: | ||
| raise errors.PluginError("Failed to resolv {0}: {1}".format(server, e)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: s/resolv/resolve/
Those unit tests will check whether name servers can be correctly resolved by certbot-dns-rfc2136 and whether requests for invalid server names or invalid IP addresses are correctly reported as errors.
zjs
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adding some more tests!
| def test_ipv4(self): | ||
| self.client("127.0.0.1") | ||
|
|
||
| # Travis Ci does not support IPv6 yet, so this test is disabled |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It has been a while since I've written Python unit tests, so this could be a crazy suggestion, but...
Would it be possible to use something like @unittest.skip("Travis CI does not support IPv6 yet, so this test is disabled") instead of just commenting it out? Hopefully this would cause the fact that the unit test is skipped to be reported in the output, to help remind anyone making changes in this area that the test is being skipped.
Even better might be to use something like @unittest.skipIf(???, "This system does not support IPv6, so this test will be skipped"), assuming there's a simple conditional check for IPv6 support you can substitute for the ???. If that works like I'm imagining, it would let people easily test IPv6 locally (on systems that support it) and automatically start testing it in Travis CI if support is ever added there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So determining whether IPv6 is supported or not in python is possible, but rather complex (and there are various ways how IPv6 might not be available, such as it is supported by the kernel but maybe just not configured on the loopback interface) and I would like to avoid this for now.
However there is an alternative solution:
@unittest.skipIf("TRAVIS" in os.environ,
"IPv6 is not always supported in Travis, IPv6 test skipped")
This will disable the IPv6 check on Travis for now. As soon as the overall Travis configuration/environment will support IPv6, we can remove this and enable the test again.
Also, I improved the general tests a bit by comparing the server property with the expected values.
Now the sucess of the DNS lookup is checked by comparing the server property with the expected value. Also the IPv6 test is only skipped when the tests are running in Travis since Travis does not always support IPv6.
|
May I support you any further with that? |
| # exception would have been raised by getaddrinfo. | ||
|
|
||
| # Just return the first address in string representation. | ||
| return results[0][4][0] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is not reliable, that is results[0][4][0] is invalid when len(results[0][4]) > 1, see below...
Test case (Windows 10),
- Install VirtualBox. (It will add multiple adaptors).
- In a console window run,
ipconfig -alland observe a number of NICs with IPv4 and IPv6 - The proposed function
_get_ip_for_hostname(server)..
a) will not reliably return an address for the NIC that is currently used for outbound network comms.
b) will only return an IPv6 address when both IPv4 and IPv6 exist
c) socket.gethostbyname(server) != _get_ip_for_hostname(server)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So, the current man-page says:
There are several reasons why the linked list may have more than one
addrinfo structure, including: the network host is multihomed, accessi‐
ble over multiple protocols (e.g., both AF_INET and AF_INET6); or the
same service is available from multiple socket types (one SOCK_STREAM
address and another SOCK_DGRAM address, for example). Normally, the
application should try using the addresses in the order in which they
are returned. The sorting function used within getaddrinfo() is
defined in RFC 3484; the order can be tweaked for a particular system
by editing /etc/gai.conf (available since glibc 2.5).
So the first element should be the best choice when you pick only a single entry.
Since it looks like we cannot get the best address for the server from the list without applying a lot of additional magic, I would recommend I change the code to return all entries. The main code will then try all entries until the DNS request succeeded for one of them. As a benefit, we can also change the configuration file in a way that you can specify multiple IP addresses there and have the additional ones as a fallback.
What do you think of that?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you should try the test case for yourself as speculation may hinder your success. For example, each item ipconfig -all returns show a valid DNS too, even if virtual. You would observe this yourself if you tested, and make a more informed design decision as to whether a test by DNS is viable (given multi DNS can exist - consider too that VMWare workstation installs a virtual DNS by default for its vNICs) ;)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should not return an address for outgoing connections but the IP address of the Nameserver where an update is needed. That should come from DNS, or whatever the resolver used to translate the hostname to a number. To Help search, one supplies the intended portnumber (ic. 53).
Also if the hostname contains an IPv4 address gethostbyname() does work, if hostname contains an IPv6 address gethostbyname() just fails.
That is the core problem is: if socket.gethostbyname() is passed an IPv6 address it does NOT copy the address to the h_addr field. like it does for IPv4 addresses, it just fails.
Therefor gethostbyname() is invalidated by reality. (in places where there is no ipv4 address available as a "backup")
So if getaddrinfo() (used by _gethostbyname(server)) cannot be used.., it would be better to drop any name lookup and just validate if it is either a valid IPv4 address or IPv6 address and create the structures themselves.
And why would socket.gethostbyname(server) return then same info as getaddrinfo(...server...) if they allow different sorting orders and getaddrinfo().. also supplies portinfo...
|
@sydneyli, said she could take a look at this. |
|
If there's any way I can help move this along, please let me know. I just ran into this same problem today and would very much prefer to use a hostname instead of a hardcoded address. |
|
@eriktews can you update this branch to resolve the merge conflicts and let the checks run again? |
|
Hi, it's done. With some luck, I can later try to improve the patch so that it will try multiple DNS servers as well. |
|
@eriktews why should it update multiple servers? |
Hi, it might happen that the name of a DNS server resolves to more than one IP address, for example one IPv6 and one IPv4 address. Maybe the system doesn't have a good IPv6 connection, so contacting the server vie IPv6 fails. When that happens, it would be a good alternative to try the IPv4 address as well. Also when there are multiple IPv4 addresses (a server cluster) and one of the servers is down, we might continue trying other addresses as well till the update succeeds with one of the servers. |
|
IMHO: |
|
Is there any reason not to use the stub resolver in |
|
Any updates here? |
|
Is there some more history here? Using a server hostname for rfc2136 worked just fine for me until recently. Is this due to an underlying library change? I found this from #7871 but am surprised that this dates back to 2018. |
|
Can any progress be made here? What is blocking this PR from landing? |
|
Why has this PR been open for over 5 years and not landed? Nothing has been done with it since I last asked about it a year ago. |
|
Facing the same prolonged inconvenience, and I see a 5-year-old pull request... Any chance to speed things up? |
The current version of certbot-dns-rfc2136 requires the user to specify an DNS server using an IP address. This commit adds support for hostnmes too.