I'm assuming the following:
- the "\\.' subpattern is there because you need to escape the backslash to get the regex engine to see the literal '.' character
- if the value is an IP address then it gets captured in one match group, and if it is a name then it gets captured in another
- as posted the pattern has unbalanced parentheses (an extra ')' in the first part and a missing ')' in the second - I would have expected you to be getting syntax errors when you ran this so I'm assuming this is a transcription error
I have also removed some of the unnecessary parentheses so as to make it clearer what you are trying to do, and I think you are close to the answer. The following works for me:
(^(([0-9]{1,2}|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]{1,2}|1[0-9]{2}|2[0-4][0-9]|25[0-5])$)|(^[a-zA-Z0-9-][a-zA-Z0-9]*[a-zA-Z0-9]$)
Now for the problems you mentioned. As written, the host name part must be at least 2 characters long and the only difference is that the firtst character can be a dash as well as the upper or lower case letter or a digit. You really need to tell us exactly what your rules are for a valid name so that we can help you with this, but if they are that the first letter must be a alphabet character and this can be followed by any number of alphanumeric or '-' characters, then
[a-zA-Z]([a-zA-Z0-9-]*)
would do (assuming that you are not wanting to use the 'ignore case' regex option ).
As written, you "host name' of 57 would match because you allow the first character to be a digit, it will skip the option middle characters and the last character can be a digit as well. Therefore 57 is considered to be perfectly valid.
You may well be having a problem with the alternation between the two parts of this pattern (the IP address and the host name) because you have used two alternation characters in a row. In effect this will allow a null string to match the pattern as a whole.
Can I suggest that you create a series of test cases that include invalid values as well as valid ones to test against. Also test each part of your pattern separately at first and make sure that they are performing as expected before joining them and re-running the entire set of tests.
By the way, your IP address subpattern allows the first value to be 0 which I don't think is valid. A small point but you have gone to some lengths to validate the numbers as well so it looks like you are not only checking that they are all digits but that the digits form valid address range values - it seems a shame to miss out on an important one.
Susan