orthanc-stone: Resources/CodeGeneration/stonegentool.py annotate

annotate Resources/CodeGeneration/stonegentool.py @ 519:17106b29ed6d bgo-commands-codegen

Changed the metadata system for structs. A __handler entry is now required (with "cpp", "ts" or both: ["cpp","ts"]). Changed the enumerations to string-based values. Adapted the integrated wasm test.

author	Benjamin Golinvaux <bgo@osimis.io>
date	Tue, 12 Mar 2019 13:11:18 +0100
parents	1dbf2d9ed1e4
children	75664eeacae5

rev	line source
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	1 import json
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	2 import yaml
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	3 import re
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	4 import os
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	5 import sys
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	6 from jinja2 import Template
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	7 from io import StringIO
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	8 import time
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	9 import datetime
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	10
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	11 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	12 1 2 3 4 5 6 7
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	13 12345678901234567890123456789012345678901234567890123456789012345678901234567890
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	14 """
3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	15
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	16 # see https://stackoverflow.com/a/2504457/2927708
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	17 def trim(docstring):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	18 if not docstring:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	19 return ''
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	20 # Convert tabs to spaces (following the normal Python rules)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	21 # and split into a list of lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	22 lines = docstring.expandtabs().splitlines()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	23 # Determine minimum indentation (first line doesn't count):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	24 indent = sys.maxsize
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	25 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	26 stripped = line.lstrip()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	27 if stripped:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	28 indent = min(indent, len(line) - len(stripped))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	29 # Remove indentation (first line is special):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	30 trimmed = [lines[0].strip()]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	31 if indent < sys.maxsize:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	32 for line in lines[1:]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	33 trimmed.append(line[indent:].rstrip())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	34 # Strip off trailing and leading blank lines:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	35 while trimmed and not trimmed[-1]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	36 trimmed.pop()
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	37 while trimmed and not trimmed[0]:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	38 trimmed.pop(0)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	39 # Return a single string:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	40 return '\n'.join(trimmed)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	41
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	42 class JsonHelpers:
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	43 """A set of utilities to perform JSON operations"""
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	44
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	45 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	46 def removeCommentsFromJsonContent(string):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	47 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	48 Remove comments from a JSON file
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	49
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	50 Comments are not allowed in JSON but, i.e., Orthanc configuration files
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	51 contains C++ like comments that we need to remove before python can
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	52 parse the file
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	53 """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	54 # remove all occurrence streamed comments (/COMMENT /) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	55 string = re.sub(re.compile("/\.?\*/", re.DOTALL), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	56
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	57 # remove all occurrence singleline comments (//COMMENT\n ) from string
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	58 string = re.sub(re.compile("//.*?\n"), "", string)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	59
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	60 return string
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	61
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	62 @staticmethod
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	63 def loadJsonWithComments(path):
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	64 """
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	65 Reads a JSON file that may contain C++ like comments
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	66 """
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	67 with open(path, "r") as fp:
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	68 fileContent = fp.read()
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	69 fileContent = JsonHelpers.removeCommentsFromJsonContent(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	70 return json.loads(fileContent)
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	71
628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	72
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	73 def LoadSchemaFromJson(filePath):
473 628941d63b8c Ongoing work. Parsing tests work bgo-osimis parents: 472 diff changeset	74 return JsonHelpers.loadJsonWithComments(filePath)
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	75
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	76 def CanonToCpp(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	77 # C++: prefix map vector and string with std::map, std::vector and
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	78 # std::string
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	79 # replace int32 by int32_t
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	80 # replace float32 by float
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	81 # replace float64 by double
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	82 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	83 retVal = retVal.replace("map", "std::map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	84 retVal = retVal.replace("vector", "std::vector")
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	85 retVal = retVal.replace("string", "std::string")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	86 retVal = retVal.replace("int32", "int32_t")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	87 retVal = retVal.replace("float32", "float")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	88 retVal = retVal.replace("float64", "double")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	89 retVal = retVal.replace("json", "Json::Value")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	90 return retVal
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	91
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	92 def CanonToTs(canonicalTypename):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	93 # TS: replace vector with Array and map with Map
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	94 # string remains string
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	95 # replace int32 by number
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	96 # replace float32 by number
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	97 # replace float64 by number
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	98 retVal = canonicalTypename
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	99 retVal = retVal.replace("map", "Map")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	100 retVal = retVal.replace("vector", "Array")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	101 retVal = retVal.replace("int32", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	102 retVal = retVal.replace("float32", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	103 retVal = retVal.replace("float64", "number")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	104 retVal = retVal.replace("bool", "boolean")
508 7105a0bad250 - Added HandleSerializedMessage to IStoneApplication (empty impl) Benjamin Golinvaux <bgo@osimis.io> parents: 507 diff changeset	105 retVal = retVal.replace("json", "Object")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	106 return retVal
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	107
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	108 def NeedsTsConstruction(enums, tsType):
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	109 if tsType == 'boolean':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	110 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	111 elif tsType == 'number':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	112 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	113 elif tsType == 'string':
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	114 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	115 else:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	116 enumNames = []
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	117 for enum in enums:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	118 enumNames.append(enum['name'])
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	119 if tsType in enumNames:
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	120 return False
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	121 return True
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	122
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	123 def NeedsCppConstruction(canonTypename):
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	124 return False
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	125
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	126 def RegisterTemplateFunction(template,func):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	127 """Makes a function callable by a jinja2 template"""
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	128 template.globals[func.__name__] = func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	129 return func
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	130
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	131 def MakeTemplate(templateStr):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	132 template = Template(templateStr)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	133 RegisterTemplateFunction(template,CanonToCpp)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	134 RegisterTemplateFunction(template,CanonToTs)
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	135 RegisterTemplateFunction(template,NeedsTsConstruction)
fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	136 RegisterTemplateFunction(template,NeedsCppConstruction)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	137 return template
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	138
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	139 def MakeTemplateFromFile(templateFileName):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	140 templateFile = open(templateFileName, "r")
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	141 templateFileContents = templateFile.read()
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	142 return MakeTemplate(templateFileContents)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	143 templateFile.close()
469 52549faf47ba Ongoing code generation work bgo-osimis parents: 468 diff changeset	144
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	145 def EatToken(sentence):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	146 """splits "A,B,C" into "A" and "B,C" where A, B and C are type names
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	147 (including templates) like "int32", "TotoTutu", or
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	148 "map<map<int32,vector<string>>,map<string,int32>>" """
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	149
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	150 if sentence.count("<") != sentence.count(">"):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	151 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	152 "Error in the partial template type list " + str(sentence) + "."
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	153 + " The number of < and > do not match!"
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	154 )
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	155
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	156 # the template level we're currently in
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	157 templateLevel = 0
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	158 for i in range(len(sentence)):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	159 if (sentence[i] == ",") and (templateLevel == 0):
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	160 return (sentence[0:i], sentence[i + 1 :])
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	161 elif sentence[i] == "<":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	162 templateLevel += 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	163 elif sentence[i] == ">":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	164 templateLevel -= 1
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	165 return (sentence, "")
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	166
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	167
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	168 def SplitListOfTypes(typename):
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	169 """Splits something like
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	170 vector<string>,int32,map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	171 in:
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	172 - vector<string>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	173 - int32
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	174 map<string,map<string,int32>>
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	175
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	176 This is not possible with a regex so
125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	177 """
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	178 stillStuffToEat = True
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	179 tokenList = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	180 restOfString = typename
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	181 while stillStuffToEat:
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	182 firstToken, restOfString = EatToken(restOfString)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	183 tokenList.append(firstToken)
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	184 if restOfString == "":
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	185 stillStuffToEat = False
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	186 return tokenList
471 125c19b294e3 Ongoing codegen work bgo-osimis parents: 470 diff changeset	187
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	188
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	189 templateRegex = \
6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	190 re.compile(r"([a-zA-Z0-9_][a-zA-Z0-9_])<([a-zA-Z0-9_,:<>]+)>")
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	191
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	192
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	193 def ParseTemplateType(typename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	194 """ If the type is a template like "SOMETHING<SOME<THING,EL<SE>>>",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	195 then it returns (true,"SOMETHING","SOME<THING,EL<SE>>")
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	196 otherwise it returns (false,"","")"""
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	197
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	198 # let's remove all whitespace from the type
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	199 # split without argument uses any whitespace string as separator
f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	200 # (space, tab, newline, return or formfeed)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	201 typename = "".join(typename.split())
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	202 matches = templateRegex.match(typename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	203 if matches == None:
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	204 return (False, "", [])
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	205 else:
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	206 m = matches
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	207 assert len(m.groups()) == 2
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	208 # we need to split with the commas that are outside of the
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	209 # defined types. Simply splitting at commas won't work
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	210 listOfDependentTypes = SplitListOfTypes(m.group(2))
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	211 return (True, m.group(1), listOfDependentTypes)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	212
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	213 def GetStructFields(struct):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	214 """This filters out the special metadata key from the struct fields"""
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	215 return [k for k in struct.keys() if k != '__handler']
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	216
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	217 def ComputeOrderFromTypeTree(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	218 ancestors,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	219 genOrder,
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	220 shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	221
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	222 if shortTypename in ancestors:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	223 raise Exception(
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	224 "Cyclic dependency chain found: the last of " + str(ancestors) +
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	225 + " depends on " + str(shortTypename) + " that is already in the list."
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	226 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	227
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	228 if not (shortTypename in genOrder):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	229 (isTemplate, _, dependentTypenames) = ParseTemplateType(shortTypename)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	230 if isTemplate:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	231 # if it is a template, it HAS dependent types... They can be
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	232 # anything (primitive, collection, enum, structs..).
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	233 # Let's process them!
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	234 for dependentTypename in dependentTypenames:
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	235 # childAncestors = ancestors.copy() NO TEMPLATE ANCESTOR!!!
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	236 # childAncestors.append(typename)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	237 ComputeOrderFromTypeTree(
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	238 ancestors, genOrder, dependentTypename, schema
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	239 )
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	240 else:
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	241 # If it is not template, we are only interested if it is a
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	242 # dependency that we must take into account in the dep graph,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	243 # i.e., a struct.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	244 if IsShortStructType(shortTypename, schema):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	245 struct = schema[GetLongTypename(shortTypename, schema)]
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	246 # The keys in the struct dict are the member names
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	247 # The values in the struct dict are the member types
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	248 if struct:
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	249 # we reach this if struct is not None AND not empty
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	250 for field in GetStructFields(struct):
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	251 # we fill the chain of dependent types (starting here)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	252 ancestors.append(shortTypename)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	253 ComputeOrderFromTypeTree(
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	254 ancestors, genOrder, struct[field], schema)
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	255 # don't forget to restore it!
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	256 ancestors.pop()
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	257
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	258 # now we're pretty sure our dependencies have been processed,
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	259 # we can start marking our code for generation (it might
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	260 # already have been done if someone referenced us earlier)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	261 if not shortTypename in genOrder:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	262 genOrder.append(shortTypename)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	263
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	264 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	265 # \| Utility functions \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	266 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	267
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	268 def IsShortStructType(typename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	269 fullStructName = "struct " + typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	270 return (fullStructName in schema)
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	271
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	272 def GetLongTypename(shortTypename, schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	273 if shortTypename.startswith("enum "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	274 raise RuntimeError('shortTypename.startswith("enum "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	275 enumName = "enum " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	276 isEnum = enumName in schema
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	277
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	278 if shortTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	279 raise RuntimeError('shortTypename.startswith("struct "):')
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	280 structName = "struct " + shortTypename
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	281 isStruct = ("struct " + shortTypename) in schema
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	282
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	283 if isEnum and isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	284 raise RuntimeError('Enums and structs cannot have the same name')
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	285
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	286 if isEnum:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	287 return enumName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	288 if isStruct:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	289 return structName
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	290
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	291 def IsTypename(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	292 return (fullName.startswith("enum ") or fullName.startswith("struct "))
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	293
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	294 def IsEnumType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	295 return fullName.startswith("enum ")
490 6470248790db ongoing codegen work bgo-osimis parents: 489 diff changeset	296
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	297 def IsStructType(fullName):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	298 return fullName.startswith("struct ")
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	299
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	300 def GetShortTypename(fullTypename):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	301 if fullTypename.startswith("struct "):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	302 return fullTypename[7:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	303 elif fullTypename.startswith("enum"):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	304 return fullTypename[5:]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	305 else:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	306 raise RuntimeError \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	307 ('fullTypename should start with either "struct " or "enum "')
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	308
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	309 def CheckSchemaSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	310 if not "rootName" in schema:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	311 raise Exception("schema lacks the 'rootName' key")
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	312 for name in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	313 if (not IsEnumType(name)) and (not IsStructType(name)) and \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	314 (name != 'rootName'):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	315 raise RuntimeError \
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	316 ('Type "' + str(name) + '" should start with "enum " or "struct "')
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	317
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	318 # TODO: check enum fields are unique (in whole namespace)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	319 # TODO: check struct fields are unique (in each struct)
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	320 # TODO: check that in the source schema, there are spaces after each colon
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	321
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	322 nonTypeKeys = ['rootName']
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	323 def GetTypesInSchema(schema):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	324 """Returns the top schema keys that are actual type names"""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	325 typeList = [k for k in schema if k not in nonTypeKeys]
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	326 return typeList
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	327
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	328 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	329 # \| Main processing logic \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	330 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	331
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	332 def ComputeRequiredDeclarationOrder(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	333 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	334 CheckSchemaSchema(schema)
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	335
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	336 # we traverse the type dependency graph and we fill a queue with
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	337 # the required struct types, in a bottom-up fashion, to compute
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	338 # the declaration order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	339 # The genOrder list contains the struct full names in the order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	340 # where they must be defined.
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	341 # We do not care about the enums here... They do not depend upon
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	342 # anything and we'll handle them, in their original declaration
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	343 # order, at the start
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	344 genOrder = []
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	345 for fullName in GetTypesInSchema(schema):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	346 if IsStructType(fullName):
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	347 realName = GetShortTypename(fullName)
8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	348 ancestors = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	349 ComputeOrderFromTypeTree(ancestors, genOrder, realName, schema)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	350 return genOrder
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	351
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	352 def GetStructFields(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	353 """Returns the regular (non __handler) struct fields"""
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	354 # the following happens for empty structs
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	355 if fieldDict == None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	356 return fieldDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	357 ret = {}
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	358 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	359 if k != "__handler":
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	360 ret[k] = v
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	361 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	362 raise RuntimeError("Fields starting with __ (double underscore) are reserved names!")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	363 return ret
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	364
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	365 def GetStructMetadata(fieldDict):
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	366 """Returns the __handler struct fields (there are default values that
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	367 can be overridden by entries in the schema
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	368 Not tested because it's a fail-safe: if something is broken in this,
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	369 dependent projects will not build."""
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	370 metadataDict = {}
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	371 metadataDict['handleInCpp'] = False
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	372 metadataDict['handleInTypescript'] = False
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	373
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	374 if fieldDict != None:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	375 for k,v in fieldDict.items():
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	376 if k.startswith("__") and k != "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	377 raise RuntimeError("Fields starting with __ (double underscore) are reserved names")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	378 if k == "__handler":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	379 if type(v) == list:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	380 for i in v:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	381 if i == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	382 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	383 elif i == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	384 metadataDict['handleInTypescript'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	385 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	386 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\"")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	387 elif type(v) == str:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	388 if v == "cpp":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	389 metadataDict['handleInCpp'] = True
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	390 elif v == "ts":
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	391 metadataDict['handleInTypescript'] = True
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	392 else:
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	393 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	394 else:
17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	395 raise RuntimeError("Error in schema. Allowed values for __handler are \"cpp\" or \"ts\" (or a list of both)")
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	396 return metadataDict
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	397
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	398 def ProcessSchema(schema, genOrder):
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	399 # sanity check
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	400 CheckSchemaSchema(schema)
472 3db3289e1c25 Ongoing codegen work bgo-osimis parents: 471 diff changeset	401
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	402 # let's doctor the schema to clean it up a bit
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	403 # order DOES NOT matter for enums, even though it's a list
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	404 enums = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	405 for fullName in schema.keys():
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	406 if IsEnumType(fullName):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	407 # convert "enum Toto" to "Toto"
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	408 typename = GetShortTypename(fullName)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	409 enum = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	410 enum['name'] = typename
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	411 assert(type(schema[fullName]) == list)
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	412 enum['fields'] = schema[fullName] # must be a list
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	413 enums.append(enum)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	414
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	415 # now that the order has been established, we actually store\
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	416 # the structs in the correct order
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	417 # the structs are like:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	418 # example = [
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	419 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	420 # "name": "Message1",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	421 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	422 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	423 # "someOtherMember":"vector<string>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	424 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	425 # },
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	426 # {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	427 # "name": "Message2",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	428 # "fields": {
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	429 # "someMember":"int32",
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	430 # "someOtherMember22":"vector<Message1>"
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	431 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	432 # }
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	433 # ]
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	434
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	435 structs = []
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	436 for i in range(len(genOrder)):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	437 # this is already the short name
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	438 typename = genOrder[i]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	439 fieldDict = schema["struct " + typename]
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	440 struct = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	441 struct['name'] = typename
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	442 struct['fields'] = GetStructFields(fieldDict)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	443 struct['__meta__'] = GetStructMetadata(fieldDict)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	444 structs.append(struct)
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	445
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	446 templatingDict = {}
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	447 templatingDict['enums'] = enums
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	448 templatingDict['structs'] = structs
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	449 templatingDict['rootName'] = schema['rootName']
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	450
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	451 return templatingDict
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	452
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	453 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	454 # \| Write to files \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	455 # +-----------------------+
474 38997ceb9bc6 Ongoing work on message code generation bgo-osimis parents: 473 diff changeset	456
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	457 # def WriteStreamsToFiles(rootName: str, genc: Dict[str, StringIO]) \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	458 # -> None:
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	459 # pass
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	460
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	461 def LoadSchema(fn):
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	462 # latin-1 is a trick, when we do NOT care about NON-ascii chars but
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	463 # we wish to avoid using a decoding error handler
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	464 # (see http://python-notes.curiousefficiency.org/en/latest/python3/text_file_processing.html#files-in-an-ascii-compatible-encoding-best-effort-is-acceptable)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	465 # TL;DR: all 256 values are mapped to characters in latin-1 so the file
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	466 # contents never cause an error.
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	467 with open(fn, 'r', encoding='latin-1') as f:
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	468 schemaText = f.read()
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	469 assert(type(schemaText) == str)
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	470 # ensure there is a space after each colon. Otherwise, dicts could be
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	471 # erroneously recognized as an array of strings containing ':'
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	472 for i in range(len(schemaText)-1):
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	473 ch = schemaText[i]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	474 nextCh = schemaText[i+1]
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	475 if ch == ':':
6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	476 if not (nextCh == ' ' or nextCh == '\n'):
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	477 lineNumber = schemaText.count("\n",0,i) + 1
519 17106b29ed6d Changed the metadata system for structs. A __handler entry is now required Benjamin Golinvaux <bgo@osimis.io> parents: 515 diff changeset	478 raise RuntimeError("Error at line " + str(lineNumber) + " in the schema: colons must be followed by a space or a newline!")
493 6fbf2eae7c88 All unit tests pass for generation, including handler and dispatcher bgo-osimis parents: 491 diff changeset	479 schema = yaml.load(schemaText)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	480 return schema
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	481
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	482 def GetTemplatingDictFromSchemaFilename(fn):
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	483 obj = LoadSchema(fn)
496 8b6ceae45ba0 Finished (untested) C++, html, typescript, tsc & browserify production. bgo-osimis parents: 494 diff changeset	484 genOrder = ComputeRequiredDeclarationOrder(obj)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	485 templatingDict = ProcessSchema(obj, genOrder)
507 ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	486 currentDT = datetime.datetime.now()
ce49eae4c887 Codegen + Warning fixes Benjamin Golinvaux <bgo@osimis.io> parents: 496 diff changeset	487 templatingDict['currentDatetime'] = str(currentDT)
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	488 return templatingDict
470 db093eb6b29d Ongoing code generation tool bgo-osimis parents: 469 diff changeset	489
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	490 # +-----------------------+
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	491 # \| ENTRY POINT \|
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	492 # +-----------------------+
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	493 def Process(schemaFile, outDir):
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	494 tdico = GetTemplatingDictFromSchemaFilename(schemaFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	495
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	496 tsTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	497 os.path.join(os.path.dirname(__file__), 'template.in.ts.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	498 template = MakeTemplateFromFile(tsTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	499 renderedTsCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	500 outputTsFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	501 outDir,str(tdico['rootName']) + "_generated.ts")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	502 with open(outputTsFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	503 outFile.write(renderedTsCode)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	504
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	505 cppTemplateFile = \
515 1dbf2d9ed1e4 Added .j2 extension to the Jinja2 template files to allow for a better syntax highlighting experience (a.o. in vscode) Benjamin Golinvaux <bgo@osimis.io> parents: 513 diff changeset	506 os.path.join(os.path.dirname(__file__), 'template.in.h.j2')
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	507 template = MakeTemplateFromFile(cppTemplateFile)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	508 renderedCppCode = template.render(**tdico)
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	509 outputCppFile = os.path.join( \
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	510 outDir, str(tdico['rootName']) + "_generated.hpp")
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	511 with open(outputCppFile,"wt",encoding='utf8') as outFile:
dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	512 outFile.write(renderedCppCode)
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	513
482 f58fe38c8c04 Ongoing work on codegen: ts and cpp enum and struct writing seem to be OK. No file write yet bgo-osimis parents: 474 diff changeset	514 if __name__ == "__main__":
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	515 import argparse
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	516
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	517 parser = argparse.ArgumentParser(
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	518 usage="""stonegentool.py [-h] [-o OUT_DIR] [-v] input_schema
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	519 EXAMPLE: python stonegentool.py -o "generated_files/" """
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	520 + """ "mainSchema.yaml,App Specific Commands.json" """
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	521 )
491 8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	522 parser.add_argument("input_schema", type=str, \
8e7e151ef472 Unit tests pass for enum generation bgo-osimis parents: 490 diff changeset	523 help="path to the schema file")
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	524 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	525 "-o",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	526 "--out_dir",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	527 type=str,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	528 default=".",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	529 help="""path of the directory where the files
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	530 will be generated. Default is current
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	531 working folder""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	532 )
772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	533 parser.add_argument(
489 f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	534 "-v",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	535 "--verbosity",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	536 action="count",
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	537 default=0,
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	538 help="""increase output verbosity (0 == errors
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	539 only, 1 == some verbosity, 2 == nerd
f6b7f113cf27 Ongoing work on code generation bgo-osimis parents: 486 diff changeset	540 mode""",
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	541 )
468 cef55b4e6c21 stonegentool first commit bgo-osimis parents: diff changeset	542
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	543 args = parser.parse_args()
494 fc17251477d6 TS and CPP tests OK. Ongoing code for C++ program that reads list of serialized messages in N files. Requires conan bgo-osimis parents: 493 diff changeset	544 schemaFile = args.input_schema
485 772516adcbf6 Ongoing work on code generation. Enums and structs OK in ts and cpp bgo-osimis parents: 482 diff changeset	545 outDir = args.out_dir
513 dea3787a8f4b Added support for struct metadata, to disable/enable handler support in Typescript or C++ Benjamin Golinvaux <bgo@osimis.io> parents: 508 diff changeset	546 Process(schemaFile, outDir)

Mercurial > hg > orthanc-stone

annotate Resources/CodeGeneration/stonegentool.py @ 519:17106b29ed6d bgo-commands-codegen