who is hereby authorized and directed to administer the same, the oath of allegiance to Her Majesty, Her Heirs and Successors; and such oath as taken and subscribed shall be transmitted by such judge to the Secretary of State of Canada, to be kept by him amongst the records of his Office.