orthanc-stone: Resources/CodeGeneration/stonegentool.py annotate

annotate Resources/CodeGeneration/stonegentool.py @ 999:2d69b8bee484

Added tests for Dicom structure set classes (loaders and utils)

author	Benjamin Golinvaux <bgo@osimis.io>
date	Fri, 20 Sep 2019 11:58:33 +0200
parents	1b47f17863ba
children

rev	line source
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	1 import json
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	2 import yaml
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	3 import re
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	4 import os
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	5 import sys
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	6 from jinja2 import Template
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	7 from io import StringIO
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	8 import time
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	9 import datetime
676 1b47f17863ba codegen: using an ordereddict loader instead of sort the keys -> the delcared ordered in the yaml is preserved in the generated code which is more meaningfull than the alphabetical order Alain Mazy <alain@mazy.be> parents: 674 diff changeset	10 import yamlloader
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	11
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	12 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	13 1 2 3 4 5 6 7
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	14 12345678901234567890123456789012345678901234567890123456789012345678901234567890
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	15 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	16
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	17 # see https://stackoverflow.com/a/2504457/2927708
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	18 def trim(docstring):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	19 if not docstring:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	20 return ''
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	21 # Convert tabs to spaces (following the normal Python rules)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	22 # and split into a list of lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	23 lines = docstring.expandtabs().splitlines()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	24 # Determine minimum indentation (first line doesn't count):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	25 indent = sys.maxsize
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	26 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	27 stripped = line.lstrip()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	28 if stripped:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	29 indent = min(indent, len(line) - len(stripped))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	30 # Remove indentation (first line is special):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	31 trimmed = [lines[0].strip()]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	32 if indent < sys.maxsize:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	33 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	34 trimmed.append(line[indent:].rstrip())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	35 # Strip off trailing and leading blank lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	36 while trimmed and not trimmed[-1]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	37 trimmed.pop()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	38 while trimmed and not trimmed[0]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	39 trimmed.pop(0)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	40 # Return a single string:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	41 return '\n'.join(trimmed)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	42
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	43 class JsonHelpers:
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	44 """A set of utilities to perform JSON operations"""
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	45
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	46 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	47 def removeCommentsFromJsonContent(string):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	48 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	49 Remove comments from a JSON file
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	50
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	51 Comments are not allowed in JSON but, i.e., Orthanc configuration files
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	52 contains C++ like comments that we need to remove before python can
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	53 parse the file
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	54 """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	55 # remove all occurrence streamed comments (/COMMENT /) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	56 string = re.sub(re.compile("/\.?\*/", re.DOTALL), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	57
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	58 # remove all occurrence singleline comments (//COMMENT\n ) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	59 string = re.sub(re.compile("//.*?\n"), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	60
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	61 return string
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	62
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	63 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	64 def loadJsonWithComments(path):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	65 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	66 Reads a JSON file that may contain C++ like comments
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	67 """
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	68 with open(path, "r") as fp:
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	69 fileContent = fp.read()
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	70 fileContent = JsonHelpers.removeCommentsFromJsonContent(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	71 return json.loads(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	72
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	73 class FieldDefinition:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	74
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	75 def __init__(self, name: str, type: str, defaultValue: str):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	76 self.name = name
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	77 self.type = type
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	78 self.defaultValue = defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	79
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	80 @staticmethod
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	81 def fromKeyValue(key: str, value: str):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	82
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	83 if "=" in value:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	84 splitValue = value.split(sep="=")
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	85 type = splitValue[0].strip(" ")
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	86 defaultValue = splitValue[1].strip(" ")
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	87 else:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	88 type = value
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	89 defaultValue = None
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	90
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	91 return FieldDefinition(name = key, type = type, defaultValue = defaultValue)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	92
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	93
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	94 def LoadSchemaFromJson(filePath):
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	95 return JsonHelpers.loadJsonWithComments(filePath)
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	96
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	97 def CanonToCpp(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	98 # C++: prefix map vector and string with std::map, std::vector and
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	99 # std::string
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	100 # replace int32... by int32_t...
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	101 # replace float32 by float
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	102 # replace float64 by double
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	103 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	104 retVal = retVal.replace("map", "std::map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	105 retVal = retVal.replace("vector", "std::vector")
543 75664eeacae5 added sets in code generation (not tested yet in TS) Alain Mazy <alain@mazy.be> parents: 519 diff changeset	106 retVal = retVal.replace("set", "std::set")
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	107 retVal = retVal.replace("string", "std::string")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	108 #uint32 and uint64 are handled by int32 and uint32 (because search and replace are done as partial words)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	109 retVal = retVal.replace("int32", "int32_t")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	110 retVal = retVal.replace("int64", "int64_t")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	111 retVal = retVal.replace("float32", "float")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	112 retVal = retVal.replace("float64", "double")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	113 retVal = retVal.replace("json", "Json::Value")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	114 return retVal
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	115
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	116 def CanonToTs(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	117 # TS: replace vector with Array and map with Map
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	118 # string remains string
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	119 # replace int32... by number
8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	120 # replace float32... by number
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	121 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	122 retVal = retVal.replace("map", "Map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	123 retVal = retVal.replace("vector", "Array")
543 75664eeacae5 added sets in code generation (not tested yet in TS) Alain Mazy <alain@mazy.be> parents: 519 diff changeset	124 retVal = retVal.replace("set", "Set")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	125 retVal = retVal.replace("uint32", "number")
8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	126 retVal = retVal.replace("uint64", "number")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	127 retVal = retVal.replace("int32", "number")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	128 retVal = retVal.replace("int64", "number")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	129 retVal = retVal.replace("float32", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	130 retVal = retVal.replace("float64", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	131 retVal = retVal.replace("bool", "boolean")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	132 retVal = retVal.replace("json", "Object")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	133 return retVal
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	134
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	135 def NeedsTsConstruction(enums, tsType):
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	136 if tsType == 'boolean':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	137 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	138 elif tsType == 'number':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	139 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	140 elif tsType == 'string':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	141 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	142 else:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	143 enumNames = []
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	144 for enum in enums:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	145 enumNames.append(enum['name'])
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	146 if tsType in enumNames:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	147 return False
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	148 return True
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	149
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	150 def NeedsCppConstruction(canonTypename):
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	151 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	152
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	153 def DefaultValueToTs(enums, field:FieldDefinition):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	154 tsType = CanonToTs(field.type)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	155
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	156 enumNames = []
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	157 for enum in enums:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	158 enumNames.append(enum['name'])
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	159
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	160 if tsType in enumNames:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	161 return tsType + "." + field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	162 else:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	163 return field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	164
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	165 def DefaultValueToCpp(root, enums, field:FieldDefinition):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	166 cppType = CanonToCpp(field.type)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	167
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	168 enumNames = []
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	169 for enum in enums:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	170 enumNames.append(enum['name'])
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	171
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	172 if cppType in enumNames:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	173 return root + "::" + cppType + "_" + field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	174 else:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	175 return field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	176
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	177 def RegisterTemplateFunction(template,func):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	178 """Makes a function callable by a jinja2 template"""
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	179 template.globals[func.__name__] = func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	180 return func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	181
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	182 def MakeTemplate(templateStr):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	183 template = Template(templateStr)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	184 RegisterTemplateFunction(template,CanonToCpp)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	185 RegisterTemplateFunction(template,CanonToTs)
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	186 RegisterTemplateFunction(template,NeedsTsConstruction)
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	187 RegisterTemplateFunction(template,NeedsCppConstruction)
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	188 RegisterTemplateFunction(template, DefaultValueToTs)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	189 RegisterTemplateFunction(template, DefaultValueToCpp)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	190 return template
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	191
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	192 def MakeTemplateFromFile(templateFileName):
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	193
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	194 with open(templateFileName, "r") as templateFile:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	195 templateFileContents = templateFile.read()
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	196 return MakeTemplate(templateFileContents)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	197
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	198
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	199 def EatToken(sentence):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	200 """splits "A,B,C" into "A" and "B,C" where A, B and C are type names
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	201 (including templates) like "int32", "TotoTutu", or
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	202 "map<map<int32,vector<string>>,map<string,int32>>" """
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	203
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	204 if sentence.count("<") != sentence.count(">"):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	205 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	206 "Error in the partial template type list " + str(sentence) + "."
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	207 + " The number of < and > do not match!"
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	208 )
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	209
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	210 # the template level we're currently in
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	211 templateLevel = 0
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	212 for i in range(len(sentence)):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	213 if (sentence[i] == ",") and (templateLevel == 0):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	214 return (sentence[0:i], sentence[i + 1 :])
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	215 elif sentence[i] == "<":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	216 templateLevel += 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	217 elif sentence[i] == ">":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	218 templateLevel -= 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	219 return (sentence, "")
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	220
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	221
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	222 def SplitListOfTypes(typename):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	223 """Splits something like
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	224 vector<string>,int32,map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	225 in:
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	226 - vector<string>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	227 - int32
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	228 map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	229
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	230 This is not possible with a regex so
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	231 """
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	232 stillStuffToEat = True
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	233 tokenList = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	234 restOfString = typename
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	235 while stillStuffToEat:
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	236 firstToken, restOfString = EatToken(restOfString)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	237 tokenList.append(firstToken)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	238 if restOfString == "":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	239 stillStuffToEat = False
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	240 return tokenList
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	241
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	242
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	243 templateRegex = \
6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	244 re.compile(r"([a-zA-Z0-9_][a-zA-Z0-9_])<([a-zA-Z0-9_,:<>]+)>")
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	245
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	246
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	247 def ParseTemplateType(typename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	248 """ If the type is a template like "SOMETHING<SOME<THING,EL<SE>>>",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	249 then it returns (true,"SOMETHING","SOME<THING,EL<SE>>")
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	250 otherwise it returns (false,"","")"""
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	251
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	252 # let's remove all whitespace from the type
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	253 # split without argument uses any whitespace string as separator
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	254 # (space, tab, newline, return or formfeed)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	255 typename = "".join(typename.split())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	256 matches = templateRegex.match(typename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	257 if matches == None:
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	258 return (False, "", [])
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	259 else:
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	260 m = matches
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	261 assert len(m.groups()) == 2
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	262 # we need to split with the commas that are outside of the
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	263 # defined types. Simply splitting at commas won't work
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	264 listOfDependentTypes = SplitListOfTypes(m.group(2))
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	265 return (True, m.group(1), listOfDependentTypes)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	266
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	267 def GetStructFields(struct):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	268 """This filters out the special metadata key from the struct fields"""
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	269 return [k for k in struct.keys() if k != '__handler']
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	270
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	271 def ComputeOrderFromTypeTree(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	272 ancestors,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	273 genOrder,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	274 shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	275
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	276 if shortTypename in ancestors:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	277 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	278 "Cyclic dependency chain found: the last of " + str(ancestors) +
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	279 + " depends on " + str(shortTypename) + " that is already in the list."
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	280 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	281
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	282 if not (shortTypename in genOrder):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	283 (isTemplate, _, dependentTypenames) = ParseTemplateType(shortTypename)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	284 if isTemplate:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	285 # if it is a template, it HAS dependent types... They can be
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	286 # anything (primitive, collection, enum, structs..).
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	287 # Let's process them!
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	288 for dependentTypename in dependentTypenames:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	289 # childAncestors = ancestors.copy() NO TEMPLATE ANCESTOR!!!
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	290 # childAncestors.append(typename)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	291 ComputeOrderFromTypeTree(
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	292 ancestors, genOrder, dependentTypename, schema
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	293 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	294 else:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	295 # If it is not template, we are only interested if it is a
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	296 # dependency that we must take into account in the dep graph,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	297 # i.e., a struct.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	298 if IsShortStructType(shortTypename, schema):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	299 struct = schema[GetLongTypename(shortTypename, schema)]
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	300 # The keys in the struct dict are the member names
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	301 # The values in the struct dict are the member types
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	302 if struct:
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	303 # we reach this if struct is not None AND not empty
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	304 for field in GetStructFields(struct):
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	305 # we fill the chain of dependent types (starting here)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	306 ancestors.append(shortTypename)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	307 ComputeOrderFromTypeTree(
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	308 ancestors, genOrder, struct[field], schema)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	309 # don't forget to restore it!
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	310 ancestors.pop()
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	311
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	312 # now we're pretty sure our dependencies have been processed,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	313 # we can start marking our code for generation (it might
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	314 # already have been done if someone referenced us earlier)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	315 if not shortTypename in genOrder:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	316 genOrder.append(shortTypename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	317
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	318 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	319 # \| Utility functions \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	320 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	321
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	322 def IsShortStructType(typename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	323 fullStructName = "struct " + typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	324 return (fullStructName in schema)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	325
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	326 def GetLongTypename(shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	327 if shortTypename.startswith("enum "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	328 raise RuntimeError('shortTypename.startswith("enum "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	329 enumName = "enum " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	330 isEnum = enumName in schema
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	331
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	332 if shortTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	333 raise RuntimeError('shortTypename.startswith("struct "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	334 structName = "struct " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	335 isStruct = ("struct " + shortTypename) in schema
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	336
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	337 if isEnum and isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	338 raise RuntimeError('Enums and structs cannot have the same name')
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	339
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	340 if isEnum:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	341 return enumName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	342 if isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	343 return structName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	344
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	345 def IsTypename(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	346 return (fullName.startswith("enum ") or fullName.startswith("struct "))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	347
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	348 def IsEnumType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	349 return fullName.startswith("enum ")
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	350
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	351 def IsStructType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	352 return fullName.startswith("struct ")
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	353
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	354 def GetShortTypename(fullTypename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	355 if fullTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	356 return fullTypename[7:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	357 elif fullTypename.startswith("enum"):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	358 return fullTypename[5:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	359 else:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	360 raise RuntimeError \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	361 ('fullTypename should start with either "struct " or "enum "')
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	362
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	363 def CheckSchemaSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	364 if not "rootName" in schema:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	365 raise Exception("schema lacks the 'rootName' key")
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	366 for name in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	367 if (not IsEnumType(name)) and (not IsStructType(name)) and \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	368 (name != 'rootName'):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	369 raise RuntimeError \
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	370 ('Type "' + str(name) + '" should start with "enum " or "struct "')
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	371
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	372 # TODO: check enum fields are unique (in whole namespace)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	373 # TODO: check struct fields are unique (in each struct)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	374 # TODO: check that in the source schema, there are spaces after each colon
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	375
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	376 nonTypeKeys = ['rootName']
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	377 def GetTypesInSchema(schema):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	378 """Returns the top schema keys that are actual type names"""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	379 typeList = [k for k in schema if k not in nonTypeKeys]
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	380 return typeList
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	381
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	382 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	383 # \| Main processing logic \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	384 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	385
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	386 def ComputeRequiredDeclarationOrder(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	387 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	388 CheckSchemaSchema(schema)
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	389
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	390 # we traverse the type dependency graph and we fill a queue with
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	391 # the required struct types, in a bottom-up fashion, to compute
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	392 # the declaration order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	393 # The genOrder list contains the struct full names in the order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	394 # where they must be defined.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	395 # We do not care about the enums here... They do not depend upon
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	396 # anything and we'll handle them, in their original declaration
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	397 # order, at the start
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	398 genOrder = []
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	399 for fullName in GetTypesInSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	400 if IsStructType(fullName):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	401 realName = GetShortTypename(fullName)
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	402 ancestors = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	403 ComputeOrderFromTypeTree(ancestors, genOrder, realName, schema)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	404 return genOrder
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	405
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	406 def GetStructFields(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	407 """Returns the regular (non __handler) struct fields"""
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	408 # the following happens for empty structs
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	409 if fieldDict == None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	410 return fieldDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	411 ret = {}
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	412 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	413 if k != "__handler":
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	414 ret[k] = FieldDefinition.fromKeyValue(k, v)
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	415 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	416 raise RuntimeError("Fields starting with __ (double underscore) are reserved names!")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	417 return ret
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	418
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	419 def GetStructMetadata(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	420 """Returns the __handler struct fields (there are default values that
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	421 can be overridden by entries in the schema
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	422 Not tested because it's a fail-safe: if something is broken in this,
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	423 dependent projects will not build."""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	424 metadataDict = {}
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	425 metadataDict['handleInCpp'] = False
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	426 metadataDict['handleInTypescript'] = False
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	427
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	428 if fieldDict != None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	429 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	430 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	431 raise RuntimeError("Fields starting with __ (double underscore) are reserved names")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	432 if k == "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	433 if type(v) == list:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	434 for i in v:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	435 if i == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	436 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	437 elif i == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	438 metadataDict['handleInTypescript'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	439 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	440 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\"")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	441 elif type(v) == str:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	442 if v == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	443 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	444 elif v == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	445 metadataDict['handleInTypescript'] = True
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	446 else:
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	447 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	448 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	449 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	450 return metadataDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	451
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	452 def ProcessSchema(schema, genOrder):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	453 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	454 CheckSchemaSchema(schema)
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	455
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	456 # let's doctor the schema to clean it up a bit
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	457 # order DOES NOT matter for enums, even though it's a list
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	458 enums = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	459 for fullName in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	460 if IsEnumType(fullName):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	461 # convert "enum Toto" to "Toto"
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	462 typename = GetShortTypename(fullName)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	463 enum = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	464 enum['name'] = typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	465 assert(type(schema[fullName]) == list)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	466 enum['fields'] = schema[fullName] # must be a list
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	467 enums.append(enum)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	468
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	469 # now that the order has been established, we actually store\
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	470 # the structs in the correct order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	471 # the structs are like:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	472 # example = [
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	473 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	474 # "name": "Message1",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	475 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	476 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	477 # "someOtherMember":"vector<string>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	478 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	479 # },
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	480 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	481 # "name": "Message2",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	482 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	483 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	484 # "someOtherMember22":"vector<Message1>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	485 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	486 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	487 # ]
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	488
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	489 structs = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	490 for i in range(len(genOrder)):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	491 # this is already the short name
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	492 typename = genOrder[i]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	493 fieldDict = schema["struct " + typename]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	494 struct = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	495 struct['name'] = typename
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	496 struct['fields'] = GetStructFields(fieldDict)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	497 struct['__meta__'] = GetStructMetadata(fieldDict)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	498 structs.append(struct)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	499
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	500 templatingDict = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	501 templatingDict['enums'] = enums
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	502 templatingDict['structs'] = structs
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	503 templatingDict['rootName'] = schema['rootName']
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	504
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	505 return templatingDict
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	506
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	507 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	508 # \| Write to files \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	509 # +-----------------------+
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	510
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	511 # def WriteStreamsToFiles(rootName: str, genc: Dict[str, StringIO]) \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	512 # -> None:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	513 # pass
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	514
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	515 def LoadSchema(fn):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	516 # latin-1 is a trick, when we do NOT care about NON-ascii chars but
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	517 # we wish to avoid using a decoding error handler
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	518 # (see http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#files-in-an-ascii-compatible-encoding-best-effort-is-acceptable)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	519 # TL;DR: all 256 values are mapped to characters in latin-1 so the file
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	520 # contents never cause an error.
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	521 with open(fn, 'r', encoding='latin-1') as f:
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	522 schemaText = f.read()
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	523 assert(type(schemaText) == str)
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	524 return LoadSchemaFromString(schemaText = schemaText)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	525
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	526 def LoadSchemaFromString(schemaText:str):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	527 # ensure there is a space after each colon. Otherwise, dicts could be
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	528 # erroneously recognized as an array of strings containing ':'
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	529 for i in range(len(schemaText)-1):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	530 ch = schemaText[i]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	531 nextCh = schemaText[i+1]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	532 if ch == ':':
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	533 if not (nextCh == ' ' or nextCh == '\n'):
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	534 lineNumber = schemaText.count("\n",0,i) + 1
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	535 raise RuntimeError("Error at line " + str(lineNumber) + " in the schema: colons must be followed by a space or a newline!")
676 1b47f17863ba codegen: using an ordereddict loader instead of sort the keys -> the delcared ordered in the yaml is preserved in the generated code which is more meaningfull than the alphabetical order Alain Mazy <alain@mazy.be> parents: 674 diff changeset	536 schema = yaml.load(schemaText, Loader = yamlloader.ordereddict.SafeLoader)
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	537 return schema
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	538
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	539 def GetTemplatingDictFromSchemaFilename(fn):
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	540 return GetTemplatingDictFromSchema(LoadSchema(fn))
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	541
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	542 def GetTemplatingDictFromSchema(schema):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	543 genOrder = ComputeRequiredDeclarationOrder(schema)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	544 templatingDict = ProcessSchema(schema, genOrder)
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	545 currentDT = datetime.datetime.now()
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	546 templatingDict['currentDatetime'] = str(currentDT)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	547 return templatingDict
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	548
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	549 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	550 # \| ENTRY POINT \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	551 # +-----------------------+
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	552 def Process(schemaFile, outDir):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	553 tdico = GetTemplatingDictFromSchemaFilename(schemaFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	554
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	555 tsTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	556 os.path.join(os.path.dirname(__file__), 'template.in.ts.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	557 template = MakeTemplateFromFile(tsTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	558 renderedTsCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	559 outputTsFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	560 outDir,str(tdico['rootName']) + "_generated.ts")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	561 with open(outputTsFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	562 outFile.write(renderedTsCode)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	563
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	564 cppTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	565 os.path.join(os.path.dirname(__file__), 'template.in.h.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	566 template = MakeTemplateFromFile(cppTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	567 renderedCppCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	568 outputCppFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	569 outDir, str(tdico['rootName']) + "_generated.hpp")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	570 with open(outputCppFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	571 outFile.write(renderedCppCode)
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	572
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	573 if __name__ == "__main__":
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	574 import argparse
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	575
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	576 parser = argparse.ArgumentParser(
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	577 usage="""stonegentool.py [-h] [-o OUT_DIR] [-v] input_schema
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	578 EXAMPLE: python stonegentool.py -o "generated_files/" """
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	579 + """ "mainSchema.yaml,App Specific Commands.json" """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	580 )
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	581 parser.add_argument("input_schema", type=str, \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	582 help="path to the schema file")
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	583 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	584 "-o",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	585 "--out_dir",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	586 type=str,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	587 default=".",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	588 help="""path of the directory where the files
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	589 will be generated. Default is current
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	590 working folder""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	591 )
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	592 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	593 "-v",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	594 "--verbosity",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	595 action="count",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	596 default=0,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	597 help="""increase output verbosity (0 == errors
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	598 only, 1 == some verbosity, 2 == nerd
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	599 mode""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	600 )
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	601
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	602 args = parser.parse_args()
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	603 schemaFile = args.input_schema
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	604 outDir = args.out_dir
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	605 Process(schemaFile, outDir)

Mercurial > hg > orthanc-stone

annotate Resources/CodeGeneration/stonegentool.py @ 999:2d69b8bee484