orthanc-stone: Resources/CodeGeneration/stonegentool.py annotate

annotate Resources/CodeGeneration/stonegentool.py @ 628:84af39146e76 am-dev

CodeGeneration: support default values

author	Alain Mazy <alain@mazy.be>
date	Wed, 08 May 2019 16:32:57 +0200
parents	8432926e9db9
children	5dd496343fad

rev	line source
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	1 import json
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	2 import yaml
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	3 import re
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	4 import os
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	5 import sys
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	6 from jinja2 import Template
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	7 from io import StringIO
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	8 import time
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	9 import datetime
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	10
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	11 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	12 1 2 3 4 5 6 7
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	13 12345678901234567890123456789012345678901234567890123456789012345678901234567890
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	14 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	15
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	16 # see https://stackoverflow.com/a/2504457/2927708
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	17 def trim(docstring):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	18 if not docstring:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	19 return ''
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	20 # Convert tabs to spaces (following the normal Python rules)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	21 # and split into a list of lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	22 lines = docstring.expandtabs().splitlines()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	23 # Determine minimum indentation (first line doesn't count):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	24 indent = sys.maxsize
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	25 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	26 stripped = line.lstrip()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	27 if stripped:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	28 indent = min(indent, len(line) - len(stripped))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	29 # Remove indentation (first line is special):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	30 trimmed = [lines[0].strip()]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	31 if indent < sys.maxsize:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	32 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	33 trimmed.append(line[indent:].rstrip())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	34 # Strip off trailing and leading blank lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	35 while trimmed and not trimmed[-1]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	36 trimmed.pop()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	37 while trimmed and not trimmed[0]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	38 trimmed.pop(0)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	39 # Return a single string:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	40 return '\n'.join(trimmed)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	41
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	42 class JsonHelpers:
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	43 """A set of utilities to perform JSON operations"""
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	44
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	45 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	46 def removeCommentsFromJsonContent(string):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	47 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	48 Remove comments from a JSON file
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	49
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	50 Comments are not allowed in JSON but, i.e., Orthanc configuration files
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	51 contains C++ like comments that we need to remove before python can
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	52 parse the file
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	53 """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	54 # remove all occurrence streamed comments (/COMMENT /) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	55 string = re.sub(re.compile("/\.?\*/", re.DOTALL), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	56
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	57 # remove all occurrence singleline comments (//COMMENT\n ) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	58 string = re.sub(re.compile("//.*?\n"), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	59
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	60 return string
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	61
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	62 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	63 def loadJsonWithComments(path):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	64 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	65 Reads a JSON file that may contain C++ like comments
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	66 """
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	67 with open(path, "r") as fp:
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	68 fileContent = fp.read()
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	69 fileContent = JsonHelpers.removeCommentsFromJsonContent(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	70 return json.loads(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	71
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	72 class FieldDefinition:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	73
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	74 def __init__(self, name: str, type: str, defaultValue: str):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	75 self.name = name
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	76 self.type = type
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	77 self.defaultValue = defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	78
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	79 @staticmethod
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	80 def fromKeyValue(key: str, value: str):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	81
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	82 if "=" in value:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	83 splitValue = value.split(sep="=")
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	84 type = splitValue[0].strip(" ")
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	85 defaultValue = splitValue[1].strip(" ")
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	86 else:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	87 type = value
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	88 defaultValue = None
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	89
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	90 return FieldDefinition(name = key, type = type, defaultValue = defaultValue)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	91
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	92
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	93 def LoadSchemaFromJson(filePath):
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	94 return JsonHelpers.loadJsonWithComments(filePath)
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	95
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	96 def CanonToCpp(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	97 # C++: prefix map vector and string with std::map, std::vector and
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	98 # std::string
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	99 # replace int32... by int32_t...
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	100 # replace float32 by float
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	101 # replace float64 by double
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	102 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	103 retVal = retVal.replace("map", "std::map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	104 retVal = retVal.replace("vector", "std::vector")
543 75664eeacae5 added sets in code generation (not tested yet in TS) Alain Mazy <alain@mazy.be> parents: 519 diff changeset	105 retVal = retVal.replace("set", "std::set")
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	106 retVal = retVal.replace("string", "std::string")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	107 #uint32 and uint64 are handled by int32 and uint32 (because search and replace are done as partial words)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	108 retVal = retVal.replace("int32", "int32_t")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	109 retVal = retVal.replace("int64", "int64_t")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	110 retVal = retVal.replace("float32", "float")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	111 retVal = retVal.replace("float64", "double")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	112 retVal = retVal.replace("json", "Json::Value")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	113 return retVal
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	114
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	115 def CanonToTs(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	116 # TS: replace vector with Array and map with Map
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	117 # string remains string
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	118 # replace int32... by number
8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	119 # replace float32... by number
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	120 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	121 retVal = retVal.replace("map", "Map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	122 retVal = retVal.replace("vector", "Array")
543 75664eeacae5 added sets in code generation (not tested yet in TS) Alain Mazy <alain@mazy.be> parents: 519 diff changeset	123 retVal = retVal.replace("set", "Set")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	124 retVal = retVal.replace("uint32", "number")
8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	125 retVal = retVal.replace("uint64", "number")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	126 retVal = retVal.replace("int32", "number")
601 8432926e9db9 codegen tools: support for int64, uint32, uint64 Alain Mazy <alain@mazy.be> parents: 543 diff changeset	127 retVal = retVal.replace("int64", "number")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	128 retVal = retVal.replace("float32", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	129 retVal = retVal.replace("float64", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	130 retVal = retVal.replace("bool", "boolean")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	131 retVal = retVal.replace("json", "Object")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	132 return retVal
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	133
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	134 def NeedsTsConstruction(enums, tsType):
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	135 if tsType == 'boolean':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	136 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	137 elif tsType == 'number':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	138 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	139 elif tsType == 'string':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	140 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	141 else:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	142 enumNames = []
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	143 for enum in enums:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	144 enumNames.append(enum['name'])
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	145 if tsType in enumNames:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	146 return False
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	147 return True
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	148
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	149 def NeedsCppConstruction(canonTypename):
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	150 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	151
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	152 def DefaultValueToTs(enums, field:FieldDefinition):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	153 tsType = CanonToTs(field.type)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	154
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	155 enumNames = []
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	156 for enum in enums:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	157 enumNames.append(enum['name'])
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	158
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	159 if tsType in enumNames:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	160 return tsType + "." + field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	161 else:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	162 return field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	163
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	164 def DefaultValueToCpp(root, enums, field:FieldDefinition):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	165 cppType = CanonToCpp(field.type)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	166
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	167 enumNames = []
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	168 for enum in enums:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	169 enumNames.append(enum['name'])
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	170
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	171 if cppType in enumNames:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	172 return root + "::" + cppType + "_" + field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	173 else:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	174 return field.defaultValue
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	175
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	176 def RegisterTemplateFunction(template,func):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	177 """Makes a function callable by a jinja2 template"""
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	178 template.globals[func.__name__] = func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	179 return func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	180
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	181 def MakeTemplate(templateStr):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	182 template = Template(templateStr)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	183 RegisterTemplateFunction(template,CanonToCpp)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	184 RegisterTemplateFunction(template,CanonToTs)
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	185 RegisterTemplateFunction(template,NeedsTsConstruction)
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	186 RegisterTemplateFunction(template,NeedsCppConstruction)
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	187 RegisterTemplateFunction(template, DefaultValueToTs)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	188 RegisterTemplateFunction(template, DefaultValueToCpp)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	189 return template
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	190
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	191 def MakeTemplateFromFile(templateFileName):
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	192
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	193 with open(templateFileName, "r") as templateFile:
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	194 templateFileContents = templateFile.read()
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	195 return MakeTemplate(templateFileContents)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	196
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	197
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	198 def EatToken(sentence):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	199 """splits "A,B,C" into "A" and "B,C" where A, B and C are type names
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	200 (including templates) like "int32", "TotoTutu", or
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	201 "map<map<int32,vector<string>>,map<string,int32>>" """
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	202
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	203 if sentence.count("<") != sentence.count(">"):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	204 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	205 "Error in the partial template type list " + str(sentence) + "."
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	206 + " The number of < and > do not match!"
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	207 )
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	208
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	209 # the template level we're currently in
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	210 templateLevel = 0
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	211 for i in range(len(sentence)):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	212 if (sentence[i] == ",") and (templateLevel == 0):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	213 return (sentence[0:i], sentence[i + 1 :])
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	214 elif sentence[i] == "<":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	215 templateLevel += 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	216 elif sentence[i] == ">":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	217 templateLevel -= 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	218 return (sentence, "")
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	219
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	220
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	221 def SplitListOfTypes(typename):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	222 """Splits something like
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	223 vector<string>,int32,map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	224 in:
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	225 - vector<string>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	226 - int32
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	227 map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	228
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	229 This is not possible with a regex so
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	230 """
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	231 stillStuffToEat = True
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	232 tokenList = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	233 restOfString = typename
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	234 while stillStuffToEat:
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	235 firstToken, restOfString = EatToken(restOfString)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	236 tokenList.append(firstToken)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	237 if restOfString == "":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	238 stillStuffToEat = False
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	239 return tokenList
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	240
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	241
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	242 templateRegex = \
6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	243 re.compile(r"([a-zA-Z0-9_][a-zA-Z0-9_])<([a-zA-Z0-9_,:<>]+)>")
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	244
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	245
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	246 def ParseTemplateType(typename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	247 """ If the type is a template like "SOMETHING<SOME<THING,EL<SE>>>",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	248 then it returns (true,"SOMETHING","SOME<THING,EL<SE>>")
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	249 otherwise it returns (false,"","")"""
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	250
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	251 # let's remove all whitespace from the type
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	252 # split without argument uses any whitespace string as separator
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	253 # (space, tab, newline, return or formfeed)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	254 typename = "".join(typename.split())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	255 matches = templateRegex.match(typename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	256 if matches == None:
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	257 return (False, "", [])
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	258 else:
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	259 m = matches
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	260 assert len(m.groups()) == 2
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	261 # we need to split with the commas that are outside of the
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	262 # defined types. Simply splitting at commas won't work
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	263 listOfDependentTypes = SplitListOfTypes(m.group(2))
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	264 return (True, m.group(1), listOfDependentTypes)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	265
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	266 def GetStructFields(struct):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	267 """This filters out the special metadata key from the struct fields"""
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	268 return [k for k in struct.keys() if k != '__handler']
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	269
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	270 def ComputeOrderFromTypeTree(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	271 ancestors,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	272 genOrder,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	273 shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	274
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	275 if shortTypename in ancestors:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	276 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	277 "Cyclic dependency chain found: the last of " + str(ancestors) +
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	278 + " depends on " + str(shortTypename) + " that is already in the list."
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	279 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	280
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	281 if not (shortTypename in genOrder):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	282 (isTemplate, _, dependentTypenames) = ParseTemplateType(shortTypename)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	283 if isTemplate:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	284 # if it is a template, it HAS dependent types... They can be
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	285 # anything (primitive, collection, enum, structs..).
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	286 # Let's process them!
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	287 for dependentTypename in dependentTypenames:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	288 # childAncestors = ancestors.copy() NO TEMPLATE ANCESTOR!!!
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	289 # childAncestors.append(typename)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	290 ComputeOrderFromTypeTree(
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	291 ancestors, genOrder, dependentTypename, schema
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	292 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	293 else:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	294 # If it is not template, we are only interested if it is a
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	295 # dependency that we must take into account in the dep graph,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	296 # i.e., a struct.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	297 if IsShortStructType(shortTypename, schema):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	298 struct = schema[GetLongTypename(shortTypename, schema)]
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	299 # The keys in the struct dict are the member names
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	300 # The values in the struct dict are the member types
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	301 if struct:
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	302 # we reach this if struct is not None AND not empty
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	303 for field in GetStructFields(struct):
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	304 # we fill the chain of dependent types (starting here)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	305 ancestors.append(shortTypename)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	306 ComputeOrderFromTypeTree(
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	307 ancestors, genOrder, struct[field], schema)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	308 # don't forget to restore it!
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	309 ancestors.pop()
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	310
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	311 # now we're pretty sure our dependencies have been processed,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	312 # we can start marking our code for generation (it might
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	313 # already have been done if someone referenced us earlier)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	314 if not shortTypename in genOrder:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	315 genOrder.append(shortTypename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	316
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	317 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	318 # \| Utility functions \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	319 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	320
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	321 def IsShortStructType(typename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	322 fullStructName = "struct " + typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	323 return (fullStructName in schema)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	324
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	325 def GetLongTypename(shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	326 if shortTypename.startswith("enum "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	327 raise RuntimeError('shortTypename.startswith("enum "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	328 enumName = "enum " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	329 isEnum = enumName in schema
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	330
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	331 if shortTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	332 raise RuntimeError('shortTypename.startswith("struct "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	333 structName = "struct " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	334 isStruct = ("struct " + shortTypename) in schema
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	335
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	336 if isEnum and isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	337 raise RuntimeError('Enums and structs cannot have the same name')
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	338
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	339 if isEnum:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	340 return enumName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	341 if isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	342 return structName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	343
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	344 def IsTypename(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	345 return (fullName.startswith("enum ") or fullName.startswith("struct "))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	346
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	347 def IsEnumType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	348 return fullName.startswith("enum ")
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	349
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	350 def IsStructType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	351 return fullName.startswith("struct ")
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	352
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	353 def GetShortTypename(fullTypename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	354 if fullTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	355 return fullTypename[7:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	356 elif fullTypename.startswith("enum"):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	357 return fullTypename[5:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	358 else:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	359 raise RuntimeError \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	360 ('fullTypename should start with either "struct " or "enum "')
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	361
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	362 def CheckSchemaSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	363 if not "rootName" in schema:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	364 raise Exception("schema lacks the 'rootName' key")
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	365 for name in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	366 if (not IsEnumType(name)) and (not IsStructType(name)) and \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	367 (name != 'rootName'):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	368 raise RuntimeError \
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	369 ('Type "' + str(name) + '" should start with "enum " or "struct "')
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	370
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	371 # TODO: check enum fields are unique (in whole namespace)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	372 # TODO: check struct fields are unique (in each struct)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	373 # TODO: check that in the source schema, there are spaces after each colon
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	374
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	375 nonTypeKeys = ['rootName']
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	376 def GetTypesInSchema(schema):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	377 """Returns the top schema keys that are actual type names"""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	378 typeList = [k for k in schema if k not in nonTypeKeys]
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	379 return typeList
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	380
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	381 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	382 # \| Main processing logic \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	383 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	384
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	385 def ComputeRequiredDeclarationOrder(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	386 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	387 CheckSchemaSchema(schema)
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	388
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	389 # we traverse the type dependency graph and we fill a queue with
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	390 # the required struct types, in a bottom-up fashion, to compute
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	391 # the declaration order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	392 # The genOrder list contains the struct full names in the order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	393 # where they must be defined.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	394 # We do not care about the enums here... They do not depend upon
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	395 # anything and we'll handle them, in their original declaration
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	396 # order, at the start
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	397 genOrder = []
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	398 for fullName in GetTypesInSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	399 if IsStructType(fullName):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	400 realName = GetShortTypename(fullName)
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	401 ancestors = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	402 ComputeOrderFromTypeTree(ancestors, genOrder, realName, schema)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	403 return genOrder
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	404
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	405 def GetStructFields(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	406 """Returns the regular (non __handler) struct fields"""
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	407 # the following happens for empty structs
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	408 if fieldDict == None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	409 return fieldDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	410 ret = {}
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	411 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	412 if k != "__handler":
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	413 ret[k] = FieldDefinition.fromKeyValue(k, v)
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	414 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	415 raise RuntimeError("Fields starting with __ (double underscore) are reserved names!")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	416 return ret
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	417
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	418 def GetStructMetadata(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	419 """Returns the __handler struct fields (there are default values that
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	420 can be overridden by entries in the schema
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	421 Not tested because it's a fail-safe: if something is broken in this,
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	422 dependent projects will not build."""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	423 metadataDict = {}
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	424 metadataDict['handleInCpp'] = False
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	425 metadataDict['handleInTypescript'] = False
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	426
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	427 if fieldDict != None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	428 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	429 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	430 raise RuntimeError("Fields starting with __ (double underscore) are reserved names")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	431 if k == "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	432 if type(v) == list:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	433 for i in v:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	434 if i == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	435 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	436 elif i == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	437 metadataDict['handleInTypescript'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	438 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	439 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\"")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	440 elif type(v) == str:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	441 if v == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	442 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	443 elif v == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	444 metadataDict['handleInTypescript'] = True
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	445 else:
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	446 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	447 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	448 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	449 return metadataDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	450
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	451 def ProcessSchema(schema, genOrder):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	452 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	453 CheckSchemaSchema(schema)
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	454
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	455 # let's doctor the schema to clean it up a bit
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	456 # order DOES NOT matter for enums, even though it's a list
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	457 enums = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	458 for fullName in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	459 if IsEnumType(fullName):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	460 # convert "enum Toto" to "Toto"
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	461 typename = GetShortTypename(fullName)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	462 enum = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	463 enum['name'] = typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	464 assert(type(schema[fullName]) == list)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	465 enum['fields'] = schema[fullName] # must be a list
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	466 enums.append(enum)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	467
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	468 # now that the order has been established, we actually store\
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	469 # the structs in the correct order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	470 # the structs are like:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	471 # example = [
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	472 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	473 # "name": "Message1",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	474 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	475 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	476 # "someOtherMember":"vector<string>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	477 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	478 # },
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	479 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	480 # "name": "Message2",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	481 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	482 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	483 # "someOtherMember22":"vector<Message1>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	484 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	485 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	486 # ]
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	487
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	488 structs = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	489 for i in range(len(genOrder)):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	490 # this is already the short name
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	491 typename = genOrder[i]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	492 fieldDict = schema["struct " + typename]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	493 struct = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	494 struct['name'] = typename
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	495 struct['fields'] = GetStructFields(fieldDict)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	496 struct['__meta__'] = GetStructMetadata(fieldDict)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	497 structs.append(struct)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	498
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	499 templatingDict = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	500 templatingDict['enums'] = enums
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	501 templatingDict['structs'] = structs
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	502 templatingDict['rootName'] = schema['rootName']
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	503
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	504 return templatingDict
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	505
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	506 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	507 # \| Write to files \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	508 # +-----------------------+
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	509
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	510 # def WriteStreamsToFiles(rootName: str, genc: Dict[str, StringIO]) \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	511 # -> None:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	512 # pass
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	513
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	514 def LoadSchema(fn):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	515 # latin-1 is a trick, when we do NOT care about NON-ascii chars but
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	516 # we wish to avoid using a decoding error handler
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	517 # (see http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#files-in-an-ascii-compatible-encoding-best-effort-is-acceptable)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	518 # TL;DR: all 256 values are mapped to characters in latin-1 so the file
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	519 # contents never cause an error.
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	520 with open(fn, 'r', encoding='latin-1') as f:
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	521 schemaText = f.read()
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	522 assert(type(schemaText) == str)
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	523 return LoadSchemaFromString(schemaText = schemaText)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	524
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	525 def LoadSchemaFromString(schemaText:str):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	526 # ensure there is a space after each colon. Otherwise, dicts could be
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	527 # erroneously recognized as an array of strings containing ':'
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	528 for i in range(len(schemaText)-1):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	529 ch = schemaText[i]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	530 nextCh = schemaText[i+1]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	531 if ch == ':':
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	532 if not (nextCh == ' ' or nextCh == '\n'):
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	533 lineNumber = schemaText.count("\n",0,i) + 1
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	534 raise RuntimeError("Error at line " + str(lineNumber) + " in the schema: colons must be followed by a space or a newline!")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	535 schema = yaml.load(schemaText)
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	536 return schema
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	537
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	538 def GetTemplatingDictFromSchemaFilename(fn):
628 84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	539 return GetTemplatingDictFromSchema(LoadSchema(fn))
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	540
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	541 def GetTemplatingDictFromSchema(schema):
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	542 genOrder = ComputeRequiredDeclarationOrder(schema)
84af39146e76 CodeGeneration: support default values Alain Mazy <alain@mazy.be> parents: 601 diff changeset	543 templatingDict = ProcessSchema(schema, genOrder)
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	544 currentDT = datetime.datetime.now()
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	545 templatingDict['currentDatetime'] = str(currentDT)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	546 return templatingDict
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	547
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	548 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	549 # \| ENTRY POINT \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	550 # +-----------------------+
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	551 def Process(schemaFile, outDir):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	552 tdico = GetTemplatingDictFromSchemaFilename(schemaFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	553
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	554 tsTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	555 os.path.join(os.path.dirname(__file__), 'template.in.ts.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	556 template = MakeTemplateFromFile(tsTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	557 renderedTsCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	558 outputTsFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	559 outDir,str(tdico['rootName']) + "_generated.ts")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	560 with open(outputTsFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	561 outFile.write(renderedTsCode)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	562
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	563 cppTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	564 os.path.join(os.path.dirname(__file__), 'template.in.h.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	565 template = MakeTemplateFromFile(cppTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	566 renderedCppCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	567 outputCppFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	568 outDir, str(tdico['rootName']) + "_generated.hpp")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	569 with open(outputCppFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	570 outFile.write(renderedCppCode)
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	571
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	572 if __name__ == "__main__":
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	573 import argparse
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	574
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	575 parser = argparse.ArgumentParser(
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	576 usage="""stonegentool.py [-h] [-o OUT_DIR] [-v] input_schema
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	577 EXAMPLE: python stonegentool.py -o "generated_files/" """
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	578 + """ "mainSchema.yaml,App Specific Commands.json" """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	579 )
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	580 parser.add_argument("input_schema", type=str, \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	581 help="path to the schema file")
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	582 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	583 "-o",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	584 "--out_dir",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	585 type=str,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	586 default=".",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	587 help="""path of the directory where the files
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	588 will be generated. Default is current
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	589 working folder""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	590 )
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	591 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	592 "-v",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	593 "--verbosity",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	594 action="count",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	595 default=0,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	596 help="""increase output verbosity (0 == errors
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	597 only, 1 == some verbosity, 2 == nerd
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	598 mode""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	599 )
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	600
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	601 args = parser.parse_args()
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	602 schemaFile = args.input_schema
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	603 outDir = args.out_dir
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	604 Process(schemaFile, outDir)

Mercurial > hg > orthanc-stone

annotate Resources/CodeGeneration/stonegentool.py @ 628:84af39146e76 am-dev