Glenn BarnasAdministrator
(KiX Supporter)
2009-01-10 10:47 PM
Finding the closest resource

I have a challenge.. I have a network with a primary (HQ) location (US-East), three regional locations (US-Central, US-West, and Canada), and about 350 branch locations. Most of the branches are connected to HQ via MPLS, and the regional locations also connect to HQ. Some of the branches connect to their closest regional site.

We have a set of data files (document templates, for example) that are being replicated to a server in every location and used by a Kix-based application. All files are maintained at the central HQ, including archives and new (development-stage) files. All active files are replicated to the regional and remote branch sites. The development files are not replicated at all, and only archives 1-year old and newer are replicated to the regional sites. (older files are pruned from the regional sites by the replication service.)

If I am at a branch, I need to be able to query my local server for the file I want. If it isn't there, I need to locate the next closest regional server, and then the master server at HQ if the data isn't in the regional server.

Since branch offices don't have "servers", a share is arbitrarily defined on the manager's workstation and the replication service is installed by the setup process.

I'm looking for thoughts/ideas on how to solve the following requirements:
  • During installation of the server/replication service on a workstation, that workstation can identify itself via some central process. The method to access the central repository, both for update and query, should be able to easily traverse subnets (and firewalls).
  • The Kix script placed on a client can not use any local configuration file or hard-coded paths to locate the data files.
  • The client should try to use the nearest server first, moving to regional and finally the primary server if it can't locate what it wants.
  • The client should never use a peer-site server (ie - a branch client cannot query a server in another branch)

I have a working solution at this moment, but A) wonder if there are better methods, and B) want to know if other admins have similar situations.

Thanks for your ideas.

Glenn

Gargoyle
(MM club member)
2009-01-11 12:36 AM
Re: Finding the closest resource

We had a similiar issue when I worked in Alaska. We got over it by defining on workstations different variables (enviromental ones) that would contain Local Resource name, Regional Resource name and HQ resource name.

As part of the installation of any new workstation, we would automatically launch a BAT file during setup, that allowed users to "program" via the batch file what enviromentals they got.

Example

Set LR = \\LocalMachine\Share
Set RR = \\RegionalServer\Share
Set HR = \\Headquaters\Share

Extremely over simplified, but it allowed us to use the %XX% in our scripts when defining drives, printers, etc...

Richard H.Administrator
(KiX Supporter)
2009-01-12 10:10 AM
Re: Finding the closest resource

Does your AD structure follow the HQ -> Region -> Branch model? If so, you could just walk up the AD structure.

If not, how about using a simple group hierarchy?

  1. A "branch" group whose description defines the locally elected branch server(s). This group is a member of:
  2. A "regional" group whose description defines the next tier of server(s). This group is a member of:
  3. An "HQ" group whose description defines the top level of server(s)


Determining the order of hosts to try by walking up the group tree.

You don't *have* to populate the branch group with all your machines - if you have a reasonable OU naming standard you can give the branch level group a name that is easy to deduce.

You can create as many tiers as you want of course.

The description which defines the server to use (or other AD field if there is something more appropriate) can be set programatically so you can do lots of tricks with it such as updating the description record when the service is installed or rotating multiple servers for simple load balancing.

Glenn BarnasAdministrator
(KiX Supporter)
2009-01-13 01:38 AM
Re: Finding the closest resource

Well, both reasonable ideas, although both rely on configuration settings that are per machine. I don't see how I could handle mobile users that travel from site to site, particularly when two sites that are physically close are in different telecom regions, thus in different regional networks. That's the real problem that I see, since AD is often split on admini-political boundaries that have little to do with the WAN connections.

Here's what I'm doing (so far) and it seems to be working out OK..

I define custom SRV records in DNS. A SRV record provides service information - a FQDN host name, service name, port number, and arbitrary weight and priority values. Take a look at your AD-DNS and you'll find SRV records representing your DCs, the PDCe, GC, LDAP, and several other network resources.

I create a custom SRV record called "_swdist" under the "_tcp" group when the application server service is installed. The installation script asks the user for the size of the network that this host serves, so a branch office (1024 addresses) would get a "22" representing 22 bits of netmask.
This value is placed in the Weight field of the SRV record, and the priority is set to 2 for a branch, 1 for a region, and 0 for the master. The one assumption is that the network is heirarichal within regions (that is, branches within a region are all subnets of the regional network.)

I use a UDF that takes 3 parameters - IP address, "Short" flag, and Type. If you specify the type, the IP and flag are ignored, and all SRV records matching the defined type are returned. This allows you to locate all DCs, GC servers, and the like. If the Short flag is true and type is null, the standard processing is short-circuited and all _SWDIST tpye records are returned without sorting or prioritizaiton. If only the IP is specified, a list of all _SWDIST records is collected. The network of each server is determined based on the Weight/Netmask value. The IP address is compared to this range and - if it matches - the record is added to a "consideration" list. After the complete list of records are processed, the Consideration list is sorted based on the number of subnet bits - a greater number being more specific (thus closest). Note that the netmask value in the SRV record is not necessarily that of the actual server's subnet - it instead represents the "supernet" of networks served.

The application processes the list, trying the closest server first. If the resource is unavailable, the next (regional) server is queried if a regional (Priority 1) server exists. Since network architecture could preclude defining regional servers by subnet, regional servers could exist without a SRV record. Before checking the master server directly for the resource, a lookup table on the master server is queried to determine if a regional server exists without a SRV record (exception process). If so that server is queried. If it doesn't exist, or does not contain the needed resource, the resource is obtained directly from the HQ Master server.

It might sound complicated, but it's actually quite simple in practice. Creation of the SRV record is automatic during the installation with just 3 questions to the installer -
1. Regional or Slave server?
2. Create SRV record?
3. Size of network served (bits) or list of CIDR networks served?

Question 3 changes based on how Q2 is answered. If a SRV record is created, you can specify a single netmask bits value. If no SRV record is created, you can specify a list of CIDR networks that this system will serve, and the central "exceptions" table is updated. Of course, if a Slave server is created, the SRV record is always created and Q2 is skipped.

With this method, I can walk into a branch with my laptop, run the application, and it will automatically connect to the local, regional, or HQ server where the resource exists. No custom configuration or settings on the client system. I can even have the app installed on a flash drive and run it without local installation - there's no software or configuration prerequisites.

I'll post my UDF and some example code in a few days. Of course, you'd need to create a few SRV records to really test the functionality, but custom SRV records can be created and deleted without any impact to the environment. For testing, I created a master on one subnet, and the regional and slave on my PC subnet (pointing to other valid PCs that had no resources). The resources can have arbitrary netmasks, unrelated to their local subnet size, so it's easy to test the heirarchy.

Glenn

Richard H.Administrator
(KiX Supporter)
2009-01-13 09:36 AM
Re: Finding the closest resource

 Quote:
Well, both reasonable ideas, although both rely on configuration settings that are per machine. I don't see how I could handle mobile users that travel from site to site


Good point. If you have a large enough peripatetic staff base then that would be a problem. If you were going down the group route then you'd base your "branch" group's name on the local (masked) subnet and create the hierarchy as before.

Glenn BarnasAdministrator
(KiX Supporter)
2009-01-29 07:02 PM
Re: Finding the closest resource

Here's the UDF I created to query the SRV records. It's also useful for locating DC, PDC, GC and similar AD records. I'll post it to the UDF forum after further testing and possibly some feedback.

Glenn


;; 
;;====================================================================== 
;; 
;;FUNCTION       GetResourceBySrv() 
;; 
;;ACTION         Returns an array of SRV record data 
;; 
;;AUTHOR         Glenn Barnas 
;; 
;;VERSION        1.0 / 2009/01/09 
;; 
;;SYNTAX         GetResourceBySrv(IpAddr [, Short] [, Type]) 
;; 
;;PARAMETERS     IpAddr - REQUIRED, IP string of host to lookup. Can be null 
;;			  if Short is true (IP is ignored) 
;; 
;;		 Short  - OPTIONAL, Flag requesting short processing. Returns 
;;			  the entire array instead of the matching hostname(s) 
;;		 	  Short is forced true if a Type value is specified. 
;; 
;;		 Type   - OPTIONAL, specifies the type of SRV record to return. The  
;;			  default type is the custom record "_swdist._tcp" (easily changed) 
;; 
;;RETURNS        Array of Arrays containing DNS SRV record data. Array format is: 
;;		     Hostname, IP, Weight, Priority, Service_Port 
;; 
;;		 By default, the UDF returns SRV records of the type "_swdist._tcp". It 
;;		 uses the Weight value as a CIDR netmasl value and only returns records 
;;		 where the IpAddr argument is in the record's subnet. It also sorts the  
;;		 data from most-specific to least specific subnet. 
;; 
;;		 If Short is True, all SRV record data is returned for external processing.  
;;		 This is useful for obtaining other SRV record types. 
;; 
;;REMARKS        In a general form, this UDF can return a set of DNS SRV records by 
;;		 specifying a Type parameter. This is useful for locating specific  
;;		 network resources, such as the PDC, all DCs, all LDAP servers, and so  
;;		 forth. In this manner, only the Type parameter is significant, and the  
;;		 UDF lends itself to many generic applications. 
;; 
;;		 In the target application, this UDF retrieves specific, custom  
;;		 SRV records ("_swdist._tcp") which identify a hierarchy of file 
;;		 server resources. There are three types of file server: 
;;		  - Master, with a priority and weight of 0 
;;		  - Secondary, with a priority of 1 and a weight of 1-31 
;;		  - Slave, with a priority of 2 and a weight of 1-31 
;;		 The slave server provides resources for a single location, the  
;;		 secondary server supports a region or scope of subnets, and the 
;;		 master is queried as a last resort when no closer resource is available. 
;;		 Data is replicated from the master to secondary and slave systems. 
;; 
;;		 The Priority value is not used by this UDF. The data returned is sorted 
;;		 by the maskbits value, so that the resource server closest to the  
;;		 client can be tried first. For example, the master site has a weight 
;;		 of 0, a secondary site has an address of 172.16.9.43 and a weight of 20, 
;;		 and the slave site has an address of 172.16.12.20 and a weight of 24. 
;;		 If a query is made with an address of 172.16.12.95, all three records 
;;		 will be returned, since that address exists in each of the three networks. 
;;		 Since the Weight values represent the maskbits, the data is sorted as 
;;		 Slave (24), Secondary (20) and then Master (0). 
;;		 The application will try to access the resource on the local Slave server. 
;;		 If it is not found, or is unavailable, the application can try the 
;;		 next resource provider (the Secondary server). Failing that, the  
;;		 Master server can be queried. 
;; 
;;		 This method of resource localization is especially useful when  
;;		 common sets of data (like software installation shares) are replicated 
;;		 from the master site out to regional offices or remote branch locations. 
;;		 With slow WAN links, the closest server is always tried first. Of course, 
;;		 you must create the SRV records that you wish to use!  
;; 
;;DEPENDENCIES   WScript 
;; 
;;TESTED WITH    W2K, WXP, W2K3, W2K8, Vista 
;; 
;;EXAMPLES       ; Get all domain controllers from local domain 
;;		 $DcList = GetResourceBySrv('', 1, '_ldap._tcp.dc._msdcs') 
;;		  
;;		 ; Get primary domain controller from local domain 
;;		 $PDCe = GetResourceBySrv('', 1, '_ldap._tcp.pdc._msdcs') 
;;		  
;;		 ; Get the list of replicated file servers closest to me 
;;		 $MyIP = Join(Split(@IPADDRESS0, ' '), '') 
;;		 $Servers = GetResourceBySrv($MyIP) 
;;		 $Tag = 0 
;;		 For Each $ServerRecord in $Servers 
;;		   If Not $Tag  
;;		     $ServerPath = '\\' + $ServerRecord[0] + '\share\folder\file.ext' 
;;		     If Exist $ServerPath 
;;		       ; open file, run command, etc.. 
;;		       If Not @ERROR 
;;		         $Tag = 1	; set Tag on successful process, prevent further attempts 
;;		       EndIf 
;;		     EndIf 
;;		   EndIf 
;;		 Next 
;;		  
;;		  
; 
Function GetResourceBySrv($_IpAddr, OPTIONAL $_Short, OPTIONAL $_Type)
 
  Dim $_, $_A			; Temp vars 
  Dim $_I, $_J, $_R		; Index pointers 
  Dim $_aR			; Return string 
  Dim $_oExec			; WScript object 
  Dim $_Rec[4]			; Working record 
  Dim $_aRtn[0]			; array of all SRV records 
  Dim $_aAdd			; array of address values 
  Dim $_DVal[33]		; array of decimal network values 
  Dim $_Network, $_Mask		; network and mask decimal values 
  Dim $_Hosts			; # of hosts in the subnet 
 
  If Not $_Type
    $_Type = '_swdist._tcp'	; default SRV record type 
  Else
    $_Short = 1			; force short processing for non-default types 
  EndIf
 
  ; Use WScript to execute the command and check the result 
  $_oExec = CreateObject("WScript.Shell").Exec('nslookup -type=all ' + $_Type)
  If Not VarType($_oExec) = 9
    $GetSrv = ''
    Exit 10
  EndIf
 
  $_J = -1			; pointer for returned data 
 
  ; get the command result and extract what is needed 
  $_aR = Split($_oExec.StdOut.ReadAll, @CRLF)
 
  ; enumerate the resulting data and place the data into the record fields 
  For $_I = 0 To UBound($_aR)
    If InStr($_aR[$_I], 'SRV service location:')
      $_ = Trim(Split($_aR[$_I + 1], '=')[1])
      $_Rec[3] = $_
      $_ = Trim(Split($_aR[$_I + 2], '=')[1])
      $_Rec[2] = $_
      $_ = Trim(Split($_aR[$_I + 3], '=')[1])
      $_Rec[4] = $_
      $_ = Trim(Split($_aR[$_I + 4], '=')[1])
      $_Rec[0] = $_
 
      ; skip over the current data and locate the A record reference to get the IP Addr 
      $_A = AScan($_aR, $_Rec[0], $_I + 5, , 1)
      $_ = Trim(Split($_aR[$_A], '=')[1])
      $_Rec[1] = $_
 
      ; update the array of SRV records 
      $_J = $_J + 1		; increment the pointer & resize the array 
      ReDim Preserve $_aRtn[$_J]
      $_aRtn[$_J] = $_Rec	; add the new data record to the array 
 
      $_I = $_I + 4		; skip the processed input data 
 
    EndIf
  Next
 
  If $_Short			; do we just return the data? 
    $GetResourceBySrv = $_aRtn
    Exit 0
  EndIf
 
 
; ===================================================================================== 
 
  ; Enumerate the SRV records, using the Weight as a CIDR netmask 
  ; Calculate the network and subnet size 
  ; order the results that match the specified network by increasing subnet size 
 
  $_R = -1			; init index pointer 
  Dim $_aWork[0]		; create working array 
  $_DVal[32] = 1.0		; init bit values 
  For $_I = 31 to 1 Step -1
    $_DVal[$_I] = $_DVal[$_I + 1] * 2.0
  Next
 
  ; Convert the supplied IP address to a double value representing the decimal address 
  $_aAdd = Split($_IpAddr, '.')
  $_IpAddr = (CDbl($_aAdd[0]) * 16777216.0) + (CDbl($_aAdd[1]) * 65536.0) + (CDbl($_aAdd[2]) * 256.0) + (CDbl($_aAdd[3]) * 1.0)
 
  ; Enumerate the SRV records in the array generated above 
  For Each $_Rec in $_aRtn
    ; Data Format: Hostname, IP, Mask, Priority, Service_Port 
 
    ; Set the number of hosts in the defined network base on the netmask 
    $_Hosts = 1.0
    For $_J = 31 to Val($_Rec[2]) Step -1
      $_Hosts = ($_Hosts * 2.0)
    Next
 
    ; convert server IP address to decimal value 
    $_aAdd = Split($_Rec[1], '.')
    $_ = (CDbl($_aAdd[0]) * 16777216.0) + (CDbl($_aAdd[1]) * 65536.0) + (CDbl($_aAdd[2]) * 256.0) + (CDbl($_aAdd[3]) * 1.0)
 
    ; Convert the Server IP address to its local network address based on the supplied maskbits 
    $_Network = 0.0
    For $_J = 1 to $_Rec[2]	; process each maskbit to value if present 
      If $_ >= CDbl($_DVal[$_J])
        $_Network = $_Network + $_DVal[$_J]
        $_ = $_ - $_DVal[$_J]	; remove the processed value from the address 
      EndIf
    Next
 
    ; If the target address is between the network start and end addresses, add the record 
    ; to the array to be returned 
    If $_IPAddr >= $_Network And $_IpAddr < ($_Network + $_Hosts)
      $_R = $_R + 1
      ReDim Preserve $_aWork[$_R]
      $_aWork[$_R] = $_Rec
    EndIf
 
  Next
 
  ; all matching records have been found. Sort the data into the return array 
  ; by largest to smallest maskbit values (closest to farthest). If two servers 
  ; have identical maskbit values, they are returned in no specific order. 
  ReDim $_aRtn[UBound($_aWork)]
  $_R = 0			; output pointer 
  For $_I = 31 to 0 Step -1	; cycle through all possible maskbit values 
    For $_J = 0 to UBound($_aWork)
      If $_aWork[$_J][2] = $_I
        $_aRtn[$_R] = $_aWork[$_J]
        $_R = $_R + 1
      EndIf
    Next
  Next
 
  $GetResourceBySrv = $_aRtn
  Exit 0
 
EndFunction