orthanc-stone: Resources/CodeGeneration/stonegentool.py annotate

annotate Resources/CodeGeneration/stonegentool.py @ 601:8432926e9db9 am-dev

codegen tools: support for int64, uint32, uint64

author	Alain Mazy <alain@mazy.be>
date	Mon, 29 Apr 2019 12:01:55 +0200
parents	75664eeacae5
children	84af39146e76

rev	line source
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	1 import json
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	2 import yaml
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	3 import re
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	4 import os
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	5 import sys
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	6 from jinja2 import Template
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	7 from io import StringIO
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	8 import time
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	9 import datetime
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	10
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	11 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	12 1 2 3 4 5 6 7
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	13 12345678901234567890123456789012345678901234567890123456789012345678901234567890
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	14 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	15
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	16 # see https://stackoverflow.com/a/2504457/2927708
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	17 def trim(docstring):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	18 if not docstring:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	19 return ''
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	20 # Convert tabs to spaces (following the normal Python rules)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	21 # and split into a list of lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	22 lines = docstring.expandtabs().splitlines()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	23 # Determine minimum indentation (first line doesn't count):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	24 indent = sys.maxsize
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	25 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	26 stripped = line.lstrip()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	27 if stripped:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	28 indent = min(indent, len(line) - len(stripped))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	29 # Remove indentation (first line is special):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	30 trimmed = [lines[0].strip()]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	31 if indent < sys.maxsize:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	32 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	33 trimmed.append(line[indent:].rstrip())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	34 # Strip off trailing and leading blank lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	35 while trimmed and not trimmed[-1]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	36 trimmed.pop()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	37 while trimmed and not trimmed[0]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	38 trimmed.pop(0)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	39 # Return a single string:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	40 return '\n'.join(trimmed)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	41
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	42 class JsonHelpers:
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	43 """A set of utilities to perform JSON operations"""
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	44
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	45 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	46 def removeCommentsFromJsonContent(string):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	47 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	48 Remove comments from a JSON file
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	49
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	50 Comments are not allowed in JSON but, i.e., Orthanc configuration files
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	51 contains C++ like comments that we need to remove before python can
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	52 parse the file
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	53 """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	54 # remove all occurrence streamed comments (/COMMENT /) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	55 string = re.sub(re.compile("/\.?\*/", re.DOTALL), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	56
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	57 # remove all occurrence singleline comments (//COMMENT\n ) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	58 string = re.sub(re.compile("//.*?\n"), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	59
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	60 return string
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	61
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	62 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	63 def loadJsonWithComments(path):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	64 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	65 Reads a JSON file that may contain C++ like comments
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	66 """
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	67 with open(path, "r") as fp:
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	68 fileContent = fp.read()
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	69 fileContent = JsonHelpers.removeCommentsFromJsonContent(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	70 return json.loads(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	71
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	72
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	73 def LoadSchemaFromJson(filePath):
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	74 return JsonHelpers.loadJsonWithComments(filePath)
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	75
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	76 def CanonToCpp(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	77 # C++: prefix map vector and string with std::map, std::vector and
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	78 # std::string
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	79 # replace int32... by int32_t...
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	80 # replace float32 by float
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	81 # replace float64 by double
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	82 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	83 retVal = retVal.replace("map", "std::map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	84 retVal = retVal.replace("vector", "std::vector")
543 75664eeacae5 added sets in code generation (not tested yet in TS) Alain Mazy <alain@mazy.be> parents: 519 diff changeset	85 retVal = retVal.replace("set", "std::set")
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	86 retVal = retVal.replace("string", "std::string")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	87 #uint32 and uint64 are handled by int32 and uint32 (because search and replace are done as partial words)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	88 retVal = retVal.replace("int32", "int32_t")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	89 retVal = retVal.replace("int64", "int64_t")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	90 retVal = retVal.replace("float32", "float")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	91 retVal = retVal.replace("float64", "double")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	92 retVal = retVal.replace("json", "Json::Value")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	93 return retVal
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	94
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	95 def CanonToTs(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	96 # TS: replace vector with Array and map with Map
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	97 # string remains string
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	98 # replace int32... by number
8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	99 # replace float32... by number
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	100 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	101 retVal = retVal.replace("map", "Map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	102 retVal = retVal.replace("vector", "Array")
543 75664eeacae5 added sets in code generation (not tested yet in TS) Alain Mazy <alain@mazy.be> parents: 519 diff changeset	103 retVal = retVal.replace("set", "Set")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	104 retVal = retVal.replace("uint32", "number")
8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	105 retVal = retVal.replace("uint64", "number")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	106 retVal = retVal.replace("int32", "number")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	107 retVal = retVal.replace("int64", "number")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	108 retVal = retVal.replace("float32", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	109 retVal = retVal.replace("float64", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	110 retVal = retVal.replace("bool", "boolean")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	111 retVal = retVal.replace("json", "Object")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	112 return retVal
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	113
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	114 def NeedsTsConstruction(enums, tsType):
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	115 if tsType == 'boolean':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	116 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	117 elif tsType == 'number':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	118 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	119 elif tsType == 'string':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	120 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	121 else:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	122 enumNames = []
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	123 for enum in enums:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	124 enumNames.append(enum['name'])
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	125 if tsType in enumNames:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	126 return False
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	127 return True
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	128
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	129 def NeedsCppConstruction(canonTypename):
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	130 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	131
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	132 def RegisterTemplateFunction(template,func):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	133 """Makes a function callable by a jinja2 template"""
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	134 template.globals[func.__name__] = func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	135 return func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	136
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	137 def MakeTemplate(templateStr):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	138 template = Template(templateStr)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	139 RegisterTemplateFunction(template,CanonToCpp)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	140 RegisterTemplateFunction(template,CanonToTs)
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	141 RegisterTemplateFunction(template,NeedsTsConstruction)
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	142 RegisterTemplateFunction(template,NeedsCppConstruction)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	143 return template
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	144
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	145 def MakeTemplateFromFile(templateFileName):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	146 templateFile = open(templateFileName, "r")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	147 templateFileContents = templateFile.read()
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	148 return MakeTemplate(templateFileContents)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	149 templateFile.close()
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	150
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	151 def EatToken(sentence):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	152 """splits "A,B,C" into "A" and "B,C" where A, B and C are type names
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	153 (including templates) like "int32", "TotoTutu", or
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	154 "map<map<int32,vector<string>>,map<string,int32>>" """
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	155
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	156 if sentence.count("<") != sentence.count(">"):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	157 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	158 "Error in the partial template type list " + str(sentence) + "."
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	159 + " The number of < and > do not match!"
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	160 )
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	161
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	162 # the template level we're currently in
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	163 templateLevel = 0
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	164 for i in range(len(sentence)):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	165 if (sentence[i] == ",") and (templateLevel == 0):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	166 return (sentence[0:i], sentence[i + 1 :])
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	167 elif sentence[i] == "<":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	168 templateLevel += 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	169 elif sentence[i] == ">":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	170 templateLevel -= 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	171 return (sentence, "")
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	172
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	173
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	174 def SplitListOfTypes(typename):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	175 """Splits something like
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	176 vector<string>,int32,map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	177 in:
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	178 - vector<string>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	179 - int32
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	180 map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	181
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	182 This is not possible with a regex so
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	183 """
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	184 stillStuffToEat = True
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	185 tokenList = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	186 restOfString = typename
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	187 while stillStuffToEat:
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	188 firstToken, restOfString = EatToken(restOfString)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	189 tokenList.append(firstToken)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	190 if restOfString == "":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	191 stillStuffToEat = False
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	192 return tokenList
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	193
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	194
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	195 templateRegex = \
6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	196 re.compile(r"([a-zA-Z0-9_][a-zA-Z0-9_])<([a-zA-Z0-9_,:<>]+)>")
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	197
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	198
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	199 def ParseTemplateType(typename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	200 """ If the type is a template like "SOMETHING<SOME<THING,EL<SE>>>",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	201 then it returns (true,"SOMETHING","SOME<THING,EL<SE>>")
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	202 otherwise it returns (false,"","")"""
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	203
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	204 # let's remove all whitespace from the type
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	205 # split without argument uses any whitespace string as separator
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	206 # (space, tab, newline, return or formfeed)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	207 typename = "".join(typename.split())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	208 matches = templateRegex.match(typename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	209 if matches == None:
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	210 return (False, "", [])
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	211 else:
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	212 m = matches
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	213 assert len(m.groups()) == 2
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	214 # we need to split with the commas that are outside of the
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	215 # defined types. Simply splitting at commas won't work
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	216 listOfDependentTypes = SplitListOfTypes(m.group(2))
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	217 return (True, m.group(1), listOfDependentTypes)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	218
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	219 def GetStructFields(struct):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	220 """This filters out the special metadata key from the struct fields"""
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	221 return [k for k in struct.keys() if k != '__handler']
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	222
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	223 def ComputeOrderFromTypeTree(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	224 ancestors,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	225 genOrder,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	226 shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	227
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	228 if shortTypename in ancestors:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	229 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	230 "Cyclic dependency chain found: the last of " + str(ancestors) +
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	231 + " depends on " + str(shortTypename) + " that is already in the list."
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	232 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	233
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	234 if not (shortTypename in genOrder):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	235 (isTemplate, _, dependentTypenames) = ParseTemplateType(shortTypename)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	236 if isTemplate:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	237 # if it is a template, it HAS dependent types... They can be
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	238 # anything (primitive, collection, enum, structs..).
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	239 # Let's process them!
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	240 for dependentTypename in dependentTypenames:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	241 # childAncestors = ancestors.copy() NO TEMPLATE ANCESTOR!!!
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	242 # childAncestors.append(typename)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	243 ComputeOrderFromTypeTree(
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	244 ancestors, genOrder, dependentTypename, schema
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	245 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	246 else:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	247 # If it is not template, we are only interested if it is a
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	248 # dependency that we must take into account in the dep graph,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	249 # i.e., a struct.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	250 if IsShortStructType(shortTypename, schema):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	251 struct = schema[GetLongTypename(shortTypename, schema)]
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	252 # The keys in the struct dict are the member names
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	253 # The values in the struct dict are the member types
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	254 if struct:
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	255 # we reach this if struct is not None AND not empty
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	256 for field in GetStructFields(struct):
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	257 # we fill the chain of dependent types (starting here)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	258 ancestors.append(shortTypename)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	259 ComputeOrderFromTypeTree(
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	260 ancestors, genOrder, struct[field], schema)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	261 # don't forget to restore it!
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	262 ancestors.pop()
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	263
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	264 # now we're pretty sure our dependencies have been processed,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	265 # we can start marking our code for generation (it might
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	266 # already have been done if someone referenced us earlier)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	267 if not shortTypename in genOrder:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	268 genOrder.append(shortTypename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	269
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	270 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	271 # \| Utility functions \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	272 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	273
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	274 def IsShortStructType(typename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	275 fullStructName = "struct " + typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	276 return (fullStructName in schema)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	277
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	278 def GetLongTypename(shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	279 if shortTypename.startswith("enum "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	280 raise RuntimeError('shortTypename.startswith("enum "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	281 enumName = "enum " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	282 isEnum = enumName in schema
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	283
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	284 if shortTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	285 raise RuntimeError('shortTypename.startswith("struct "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	286 structName = "struct " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	287 isStruct = ("struct " + shortTypename) in schema
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	288
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	289 if isEnum and isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	290 raise RuntimeError('Enums and structs cannot have the same name')
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	291
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	292 if isEnum:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	293 return enumName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	294 if isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	295 return structName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	296
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	297 def IsTypename(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	298 return (fullName.startswith("enum ") or fullName.startswith("struct "))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	299
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	300 def IsEnumType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	301 return fullName.startswith("enum ")
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	302
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	303 def IsStructType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	304 return fullName.startswith("struct ")
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	305
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	306 def GetShortTypename(fullTypename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	307 if fullTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	308 return fullTypename[7:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	309 elif fullTypename.startswith("enum"):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	310 return fullTypename[5:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	311 else:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	312 raise RuntimeError \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	313 ('fullTypename should start with either "struct " or "enum "')
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	314
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	315 def CheckSchemaSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	316 if not "rootName" in schema:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	317 raise Exception("schema lacks the 'rootName' key")
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	318 for name in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	319 if (not IsEnumType(name)) and (not IsStructType(name)) and \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	320 (name != 'rootName'):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	321 raise RuntimeError \
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	322 ('Type "' + str(name) + '" should start with "enum " or "struct "')
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	323
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	324 # TODO: check enum fields are unique (in whole namespace)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	325 # TODO: check struct fields are unique (in each struct)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	326 # TODO: check that in the source schema, there are spaces after each colon
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	327
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	328 nonTypeKeys = ['rootName']
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	329 def GetTypesInSchema(schema):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	330 """Returns the top schema keys that are actual type names"""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	331 typeList = [k for k in schema if k not in nonTypeKeys]
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	332 return typeList
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	333
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	334 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	335 # \| Main processing logic \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	336 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	337
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	338 def ComputeRequiredDeclarationOrder(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	339 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	340 CheckSchemaSchema(schema)
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	341
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	342 # we traverse the type dependency graph and we fill a queue with
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	343 # the required struct types, in a bottom-up fashion, to compute
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	344 # the declaration order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	345 # The genOrder list contains the struct full names in the order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	346 # where they must be defined.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	347 # We do not care about the enums here... They do not depend upon
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	348 # anything and we'll handle them, in their original declaration
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	349 # order, at the start
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	350 genOrder = []
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	351 for fullName in GetTypesInSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	352 if IsStructType(fullName):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	353 realName = GetShortTypename(fullName)
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	354 ancestors = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	355 ComputeOrderFromTypeTree(ancestors, genOrder, realName, schema)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	356 return genOrder
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	357
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	358 def GetStructFields(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	359 """Returns the regular (non __handler) struct fields"""
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	360 # the following happens for empty structs
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	361 if fieldDict == None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	362 return fieldDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	363 ret = {}
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	364 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	365 if k != "__handler":
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	366 ret[k] = v
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	367 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	368 raise RuntimeError("Fields starting with __ (double underscore) are reserved names!")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	369 return ret
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	370
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	371 def GetStructMetadata(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	372 """Returns the __handler struct fields (there are default values that
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	373 can be overridden by entries in the schema
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	374 Not tested because it's a fail-safe: if something is broken in this,
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	375 dependent projects will not build."""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	376 metadataDict = {}
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	377 metadataDict['handleInCpp'] = False
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	378 metadataDict['handleInTypescript'] = False
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	379
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	380 if fieldDict != None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	381 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	382 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	383 raise RuntimeError("Fields starting with __ (double underscore) are reserved names")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	384 if k == "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	385 if type(v) == list:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	386 for i in v:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	387 if i == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	388 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	389 elif i == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	390 metadataDict['handleInTypescript'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	391 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	392 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\"")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	393 elif type(v) == str:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	394 if v == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	395 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	396 elif v == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	397 metadataDict['handleInTypescript'] = True
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	398 else:
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	399 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	400 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	401 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	402 return metadataDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	403
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	404 def ProcessSchema(schema, genOrder):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	405 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	406 CheckSchemaSchema(schema)
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	407
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	408 # let's doctor the schema to clean it up a bit
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	409 # order DOES NOT matter for enums, even though it's a list
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	410 enums = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	411 for fullName in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	412 if IsEnumType(fullName):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	413 # convert "enum Toto" to "Toto"
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	414 typename = GetShortTypename(fullName)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	415 enum = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	416 enum['name'] = typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	417 assert(type(schema[fullName]) == list)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	418 enum['fields'] = schema[fullName] # must be a list
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	419 enums.append(enum)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	420
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	421 # now that the order has been established, we actually store\
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	422 # the structs in the correct order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	423 # the structs are like:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	424 # example = [
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	425 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	426 # "name": "Message1",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	427 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	428 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	429 # "someOtherMember":"vector<string>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	430 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	431 # },
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	432 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	433 # "name": "Message2",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	434 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	435 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	436 # "someOtherMember22":"vector<Message1>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	437 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	438 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	439 # ]
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	440
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	441 structs = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	442 for i in range(len(genOrder)):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	443 # this is already the short name
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	444 typename = genOrder[i]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	445 fieldDict = schema["struct " + typename]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	446 struct = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	447 struct['name'] = typename
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	448 struct['fields'] = GetStructFields(fieldDict)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	449 struct['__meta__'] = GetStructMetadata(fieldDict)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	450 structs.append(struct)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	451
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	452 templatingDict = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	453 templatingDict['enums'] = enums
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	454 templatingDict['structs'] = structs
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	455 templatingDict['rootName'] = schema['rootName']
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	456
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	457 return templatingDict
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	458
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	459 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	460 # \| Write to files \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	461 # +-----------------------+
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	462
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	463 # def WriteStreamsToFiles(rootName: str, genc: Dict[str, StringIO]) \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	464 # -> None:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	465 # pass
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	466
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	467 def LoadSchema(fn):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	468 # latin-1 is a trick, when we do NOT care about NON-ascii chars but
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	469 # we wish to avoid using a decoding error handler
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	470 # (see http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#files-in-an-ascii-compatible-encoding-best-effort-is-acceptable)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	471 # TL;DR: all 256 values are mapped to characters in latin-1 so the file
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	472 # contents never cause an error.
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	473 with open(fn, 'r', encoding='latin-1') as f:
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	474 schemaText = f.read()
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	475 assert(type(schemaText) == str)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	476 # ensure there is a space after each colon. Otherwise, dicts could be
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	477 # erroneously recognized as an array of strings containing ':'
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	478 for i in range(len(schemaText)-1):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	479 ch = schemaText[i]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	480 nextCh = schemaText[i+1]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	481 if ch == ':':
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	482 if not (nextCh == ' ' or nextCh == '\n'):
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	483 lineNumber = schemaText.count("\n",0,i) + 1
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	484 raise RuntimeError("Error at line " + str(lineNumber) + " in the schema: colons must be followed by a space or a newline!")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	485 schema = yaml.load(schemaText)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	486 return schema
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	487
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	488 def GetTemplatingDictFromSchemaFilename(fn):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	489 obj = LoadSchema(fn)
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	490 genOrder = ComputeRequiredDeclarationOrder(obj)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	491 templatingDict = ProcessSchema(obj, genOrder)
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	492 currentDT = datetime.datetime.now()
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	493 templatingDict['currentDatetime'] = str(currentDT)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	494 return templatingDict
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	495
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	496 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	497 # \| ENTRY POINT \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	498 # +-----------------------+
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	499 def Process(schemaFile, outDir):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	500 tdico = GetTemplatingDictFromSchemaFilename(schemaFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	501
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	502 tsTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	503 os.path.join(os.path.dirname(__file__), 'template.in.ts.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	504 template = MakeTemplateFromFile(tsTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	505 renderedTsCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	506 outputTsFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	507 outDir,str(tdico['rootName']) + "_generated.ts")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	508 with open(outputTsFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	509 outFile.write(renderedTsCode)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	510
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	511 cppTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	512 os.path.join(os.path.dirname(__file__), 'template.in.h.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	513 template = MakeTemplateFromFile(cppTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	514 renderedCppCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	515 outputCppFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	516 outDir, str(tdico['rootName']) + "_generated.hpp")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	517 with open(outputCppFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	518 outFile.write(renderedCppCode)
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	519
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	520 if __name__ == "__main__":
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	521 import argparse
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	522
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	523 parser = argparse.ArgumentParser(
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	524 usage="""stonegentool.py [-h] [-o OUT_DIR] [-v] input_schema
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	525 EXAMPLE: python stonegentool.py -o "generated_files/" """
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	526 + """ "mainSchema.yaml,App Specific Commands.json" """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	527 )
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	528 parser.add_argument("input_schema", type=str, \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	529 help="path to the schema file")
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	530 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	531 "-o",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	532 "--out_dir",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	533 type=str,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	534 default=".",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	535 help="""path of the directory where the files
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	536 will be generated. Default is current
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	537 working folder""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	538 )
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	539 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	540 "-v",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	541 "--verbosity",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	542 action="count",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	543 default=0,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	544 help="""increase output verbosity (0 == errors
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	545 only, 1 == some verbosity, 2 == nerd
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	546 mode""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	547 )
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	548
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	549 args = parser.parse_args()
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	550 schemaFile = args.input_schema
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	551 outDir = args.out_dir
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	552 Process(schemaFile, outDir)

Mercurial > hg > orthanc-stone

annotate Resources/CodeGeneration/stonegentool.py @ 601:8432926e9db9 am-dev