up9inc / mockintosh Goto Github PK

Hi,

I tried to deploy mockintosh onto our test windows server 2012 to replace other mocking tools but it threw the following error.
Commented out SIGHUP lines in init.py and it works. Wondering if should make the SIGHUP handling optional or configurable.

C:\Users\ryan.song\Desktop>mockintosh mock.yaml
[2021-12-07 22:24:32,701 root INFO] Mockintosh v0.13.17 is starting...
[2021-12-07 22:24:32,701 root INFO] Reading configuration file from path: mock.y
aml
[2021-12-07 22:24:32,717 root INFO] Configuration file is a valid YAML file.
[2021-12-07 22:24:32,717 root INFO] Configuration file is valid according to the
JSON schema.
Traceback (most recent call last):
File "runpy.py", line 194, in run_module_as_main
File "runpy.py", line 87, in run_code
File "C:\Users\ryan.song\AppData\Local\Mockintosh\bin\mockintosh.exe_main.
py", line 18, in
File "C:\Users\ryan.song\AppData\Local\Mockintosh\pkgs\mockintosh_init.py"
, line 288, in initiate
while run(source, debug=debug_mode, interceptors=interceptors, address=addre
ss,
File "C:\Users\ryan.song\AppData\Local\Mockintosh\pkgs\mockintosh_init_.py"
, line 115, in run
prev_handler = signal.getsignal(signal.SIGHUP)
AttributeError: module 'signal' has no attribute 'SIGHUP'

Host Name: WIN-PMVEPODMBCO
OS Name: Microsoft Windows Server 2012 R2 Standard
OS Version: 6.3.9600 N/A Build 9600
OS Manufacturer: Microsoft Corporation
OS Configuration: Member Server
OS Build Type: Multiprocessor Free
Registered Owner: EC2
Registered Organization: Amazon.com
Original Install Date: 8/30/2019, 7:13:39 PM
System Boot Time: 11/13/2021, 5:02:33 PM
System Manufacturer: Xen
System Model: HVM domU
System Type: x64-based PC
Processor(s): 1 Processor(s) Installed.
[01]: Intel64 Family 6 Model 79 Stepping 1 GenuineInt
el ~2300 Mhz
BIOS Version: Xen 4.2.amazon, 8/24/2006
Windows Directory: C:\Windows
System Directory: C:\Windows\system32
Boot Device: \Device\HarddiskVolume1
System Locale: en-us;English (United States)
Input Locale: en-us;English (United States)
Time Zone: (UTC) Coordinated Universal Time
Total Physical Memory: 8,192 MB
Available Physical Memory: 2,767 MB
Virtual Memory: Max Size: 16,384 MB
Virtual Memory: Available: 10,039 MB
Virtual Memory: In Use: 6,345 MB
Page File Location(s): C:\pagefile.sys

I think it would be a good thing to be able to use schema registry to serialize/deserialize messages produced/consumed by async actors.

I don't know if there is already a recommended way to deal with this. At some point, I thought I could achieve this with interceptors but upon closer inspection, it seems they are meant for HTTP interactions. Maybe we should implement something analogous to this interceptor concept but for producers and consumers.

Environment:
mockintosh==0.13.17
Docker version 20.10.12, build e91ed57
MacOS Monterey v.12.2.1

Executing mockintosh with non-relative cert paths result in a Certificate loading error.

Expectation:
Specifying a full path to a cert file should work.

Actual:

mockintosh.exceptions.CertificateLoadingError:
Certificate loading error: Path `/my/docker/volume/config/certs/mockingjay.crt` is inaccessible!

Relative Path Restricting code:

Hello, I am using reactive producer for kafka mock actor as described here in documentation

The variables from consumer are not present in producer

here is a dummy code for actors configuration that I am using :

      - name: dummy-mock
        consume:
          queue: REQUEST
          capture: 10
          schema: "@value/Request.json"
        produce:
          queue: RESPONSE
          key: "can reference as {{consumed.value}}"
          value: "@value/Response.json"

here is Request.json:

{
  "someId": "{{someId}}",
  "code": "{{code}}"
}

here is Response.json:

{
  "someId": "{{someId}}",
  "code": "{{code}}",
  "someOtherField": "someValue"
}

after producing a message like:

{
  "someId": "someValue",
  "code": "someValue"
}

the consumer receives message, producer produces a message too but I get following warning:

[2022-08-18 12:41:47,241 root WARNING] Handlebars: Could not find variable 'someId'

and the produced message is without filled values :

{
  "someId": "{{someId}}",
  "code": "{{code}}",
  "someOtherField": "someValue"
}

As we expand on our usage of mockintosh we need the ability to provide custom template methods to do more complicated actions. For instance, we have a use-case where we want to inject certain complicated structures into objects based on some conditions.

FWIW we are using the docker container and happy to build out new python methods. We just need an easy way to wire them into Mockintosh. At the moment we are mounting copies of the hbs/methods.py and templating.py files into the container with custom code.

I have limited python knowledge but i think something like the following could be useful.

1. Mockintosh uses something to determine where to read custom methods.
   a. An env var to define where custom methods files will be read. MOCKINTOSH_TEMPLATE_METHODS_DIR.
   b. OR additional yaml configuration that specifies either a directory or specific files to load?
2. Place the file with custom methods at that ^^ location.
3. Each methods file will expose an inject_methods() -> list[Callable] method. These defined methods can be merged with the existing methods in the templating.py file.

We have a use-case where we are using the reactive consumer and want to alter the JSON emitted from the producer.

Therefore i'd propose adding a fromjson method to mirror the tojson method. This gives templates more flexibility in how it deals with data.

It's been over a year since last release. If I've understood correctly, in the next one some critical issues should be fixed, e.g. #158

Isn't it time for a new release, maybe?

I enabled Traffic Log in the management UI. When I observed in the Network tab from Developer Tools, it keeps sending DELETE calls for every second. Is this expected behavior?

I used the following configuration for kafka

management:
  port: 8000

services:
  - name: Kafka Mock Actors
    type: kafka
    address: "{{env 'KAFKA'}}"
    actors:
      - name: span
        produce:
          create: true
          queue: jaeger-spans
          value: "@/span1.json"

But when I triggered the produce I got the following error in logs

%3|1671090820.157|FAIL|rdkafka#producer-1| [thrd:{{env 'KAFKA'}}:9092/bootstrap]: {{env 'KAFKA'}}:9092/bootstrap: Failed to resolve '{{env 'KAFKA'}}:9092': Name or service not known (after 0ms in state CONNECT, 28 identical error(s) suppressed)

I have installed Mockintosh as a Python package in Pycharm in Windows 10 Pro. After configuring the new service, the terminal output shows that Mock server is ready. But when I launch the localhost, it is not showing any response.

OS: Windows 10 Pro
Python version: 3.9

service.yaml

services:
  - name: capi
    port: 9001
    hostname: localhost
    endpoints:
      - path: /
        method: GET
        response: 'Hello World'

Terminal Output

[2021-03-11 22:13:42,024 root DEBUG] Configuration text: services:
  - name: capi
    port: 9001
    hostname: localhost
    endpoints:
      - path: /
        method: GET
        response: 'Hello World'
[2021-03-11 22:13:42,025 root INFO] Configuration file is a valid YAML file.
[2021-03-11 22:13:42,028 root INFO] Configuration file is valid according to the JSON schema.
[2021-03-11 22:13:42,029 root DEBUG] Templating engine (config) is: Handlebars
[2021-03-11 22:13:42,033 root INFO] Mock server is ready!
[2021-03-11 22:13:42,034 asyncio DEBUG] Using proactor: IocpProactor

`management:
port: 8000
services:

• port: 8001
  name: Mock Server
  endpoints:
  • path: /create-user
    method: POST
    headers:
    Accept: application/json
    body:
    schema:
    type: object
    properties:
    name:
    type: string
    instanceId:
    type: string
    id:
    type: string
    response:
    status: 200
    headers:
    content-type: application/json
    body: '{ "name": {{ body.schema.properties.name }}, "id": {{ body.schema.properties.id }}, "message": "User created" }'`

I have tried above post method moch with body data as object., Kindly check and update if any configuration is missing.

I'm using up9inc/mockintosh docker image to run mockintosh server. When I try to stop the running docker container, it will not stop immediately. Instead, docker will wait for around 10 seconds and then force-kill it, which is pretty annoying.

I think the reason is that mockintosh entry point does not react to SIGTERM signal: if I exec bash in the running container and run kill 1 there, nothing happens, even though that command is expected to terminate the mockintosh process.

Hi @undera @mertyildiran,
Thanks for having such an wondeful feature of mocking rest api and kakfa producer/consumer events. I came across a sitaution in the current project that I'm working where my current application calling a rest api which sends 200 back in response and also posts an event to a kafka topic. The kafka event is also consumed back again by the application.

I would like to suggest an new feature where we have an mock response as step 1 and publish an event to the kafka topic as a subsequent event.

YAML based OpenAPI spec - when I tried to convert to mockintosh mocks, I found that it provides only URLs and Parameters. Response, request body etc is lost. Please fix this quickly.

Hi,

Thanks for the awesome tool!

I have a minor issue regarding loading configuration file.
What current "Load" button on management UI does is actually "reset" config in memory instead of reloading configuration file from disk.
What I'm looking for is a button to reload file from disk so that don't need to restart mockintosh every time the config file is updated.
Could you please consider "fixing" the load button or adding a new button? Thank you!

Hello guys.

I have a use case where the client calls an HTTP endpoint, receives a 202 status code, and the real response payload is delivered asynchronously through a queue. So far I have been able to model this use case in mockintosh by using an async producer and triggering it in my endpoint with triggerAsyncProducer.

My problem right now is that the full use case requires the endpoint to return a correlation-id to the caller, and the async message delivered later must reference this same correlation-id.

I almost made it work by using the counters feature, with a config like this:

services:
- name: Correlation Service
  endpoints:
  - path: /correlation
    response:
      status: 202
      body: '{"correlationId": "{{counter ''correlationIdCounter''}}"}'
      triggerAsyncProducer: rabbit-producer
- name: Rabbit Service
  type: amqp
  address: localhost:5672
  actors:
  - name: rabbit-producer
    produce:
      queue: mockintosh
      key: mockintosh
      value: '{"message": "ok", "correlationId": "{{correlationIdCounter}}"}'

My problem right now is that it seems that the async message is produced BEFORE the counter has been incremented.
So in the first execution, the http endpoint correctly gives me a correlation-id of 1, but I receive this in my queue:

{"message": "ok", "correlationId": "{{correlationIdCounter}}"}

For the next executions, the http endpoint correctly returns the incremented counter, but the produced message is always one step behind.

I don't know if this is a bug or the expected behavior.
If this is expected, is there any other way to make my use case work?

One idea that came to my mind is this: mockintosh keeps a count of the requests as we can see on the Statistics tab. If there's a way to reference this count in the templated responses (something like {{requestCount}}), then I could also use this count to model my correlation-id requirement.

I was thinking about adding some conditionals to check if the counter exists (first execution) and then always adding 1 to it's value but I haven't tried it yet, don't even know if it's possible.

Thanks!

The OpenAPI support is experimental, so it isnt surprising that OpenAPI spec callbacks are not supported yet.

OpenAPI callbacks define outgoing async requests, typically back to the sender.
docs: https://swagger.io/docs/specification/callbacks/
spec: https://spec.openapis.org/oas/v3.0.2#callbackObject

https://blog.stoplight.io/mocking-callbacks-openapi-prism shows how these callbacks are utilised by https://github.com/stoplightio/prism

It would be great if the OpenAPI support included support for these in the same manner.

It looks like the mockintosh native config yaml also does not support this type of async HTTP callback, but I would love to be proven wrong about that.

In order to use authentication for Kafka additional parameters are required. Since the implementation uses confluent-kafka based on librdkafka the parameters are known to people exposed to Kafka.

I did a quick POC about a quick way to approach this but it requires feedback.

Hi. I was just wondering if there is plan for openapi support. It would be great if you could just use openapi3.x specification and based on that create mocked service with endpoint responsing according to what is specified in openapi yaml file.

We have some team members that are on M1 macs. Unfortunately M1 macs either have to run AMD images in emulation or out-right cannot use them. Therefore distributing an ARM version of mockintosh's docker image would be fantastic.

https://docs.docker.com/desktop/multi-arch/

Hello!

I'm sorry if this has been answered before, but is it possible to generate arrays from templates?

I.e. I have a template looking like this:
device.json.j2:

'{ 
"name" : "{{ fake.last_name() }}",
"address" : "{{ fake.ipv4() }}",
"port" : "{{ random.int 1 65536 }}",
}'

And inside a response I'd like to retrieve an array of devices with variable length. I.e. I'd like to mock when an endpoint returns a collection of devices.

My attempt so far looks like this (note the multiple template listings, as I've been unable to figure out how to tell the array to generate X amount of templated devices):

mockconf.yaml

- path: /pnds/{{ pndUuid }}/devices
    method: GET
    response:
    - status: 200
      content-type: application/json
      body:
        {{ array '@templates/list_entity.json.j2' '@templates/list_entity.json.j2' '@templates/list_entity.json.j2' }}

My intention is it, to have an endpoint generate a bunch of these devices, to mock returning collections of devices from the endpoint.

Traffic Log enable options in the document is not clear. Please let me know how do we enable the traffic log.

Hi,

I got the following error when trying to install with brew

Last 15 lines from /Users/schmil/Library/Logs/Homebrew/mockintosh/10.pip:
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/local/Cellar/mockintosh/0.13.17/libexec/bin/python3.9 /usr/local/Cellar/mockintosh/0.13.17/libexec/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /private/tmp/tmp6am54efc
  cwd: /private/tmp/mockintosh--graphql-core-20221211-22175-18oruqj/graphql-core-3.1.6
  Preparing metadata (pyproject.toml): finished with status 'error'
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Do not report this issue to Homebrew/brew or Homebrew/core!```

The policy of our company blocks the installation of a unsigned tool.
Please add a signed certificate to the msi installer for Windows.

In case of many services being mocked - it would be easier to keep configs for each service in separate file.
like this:
example-service-1.yaml

- name: example-service-1
      port: 80
      endpoints:
        - path: ...

example-service-2.yaml

- name: example-service-2
      port: 80
      endpoints:
        - path: ...

I haven't found an answer to this question in the docs or the webinar. Please point me to the docs/examples if any.

If a service talks to a third party dependency using HTTPS and I would like to use mockintosh to mock the responses from this third party dependency, how would I do this when I want to build templates based on unhandled requests instead of carefully crafting them manually?

Hi 👋

Is it possible to base64 encode via response templates?
Something like:

body: "{{ 'some_token' | b64encode }}"

This can slightly help in mocks maintenance.

When I modify the key-value pair order in the YAML in Configuration page, it is not applying the YAML as it is losing the YAML format in the request payload.

Initial YAML

services:
- name: Search Service
  performanceProfile: loadprofile
  hostname: localhost
  port: 8001
  endpoints:
  - path: /

Modified YAML in the Configuration page

services:
- name: Search Service
  port: 8001
  performanceProfile: loadprofile
  hostname: localhost
  endpoints:
  - path: /

After hitting Applying Changes. Here is the output:

Request Payload

Hello, I just spent many hours trying to make the Amazon SQS integration work.
I found some documentation and code issues that I'd like to share so we can make Mockintosh a better product.

1️⃣ Installing the "cloud" package

The documentation tells you to install the mockintosh[cloud] package to get the boto3 dependency. Copying/pasting the command shown on the documentation did not work on my mac, it only worked when I used quotes around the package name like this: pip3 install "mockintosh[cloud]".

Also, it is not clear if the package name is really mockintosh[cloud] or if it should be mockintosh[aws]. I tried both and none of them gave me an error.

2️⃣ Problem with the "markupsafe" dependency

After installing the cloud package like above, Mockintosh would no longer start and gave me this error:

ImportError: cannot import name 'soft_unicode' from 'markupsafe'

It could be a problem with one of the AWS dependencies or maybe Jinja2, there are issues like this one mentioning it.
Anyway, I had to force a downgrade of MarkupSafe with pip3 install MarkupSafe==2.0.1 to make it work.

3️⃣ Using credentials (like secret_access_key) with special characters in them

My secret access key has a slash in it, just like this: gvmooa/xxxxxxxxx.
This messes up the "address" field of the configuration file and some parts of the code that use urlparse will get wrong values, like this one.

I see two ways to solve this:

1. The way I solved it: just use the environment variables that are already mentioned in the documentation, and remove the credentials from the address field. However, the documentation should make it clear that this is the only way to work with credentials that have special characters like /, :, ?, @ or #.

2. Change the implementation (and make it clear in the documentation) so we can provide url encoded credentials in the address field. Eg: we would use %2F instead of / in the address and urllib.parse will work correctly. However, you will also need to change the implementation to unquote these values:

# import unquote: from urllib.parse import unquote
# in the _get_resource method:
aws_secret_access_key=environ.get('AWS_SECRET_ACCESS_KEY', '' if parsed.password is None else unquote(parsed.password)),
aws_access_key_id=environ.get('AWS_ACCESS_KEY_ID', '' if parsed.username is None else unquote(parsed.username)),

4️⃣ Queue must be FIFO and named with a ".fifo" suffix.

The last struggle was trying to understand why I was getting errors telling me that the queue was not found.
Bear in mind that none of this is mentioned in the documentation. 🤦

I had to look at your source code (this line) to understand that you expect the queue name to have a .fifo suffix in AWS, but not in the configuration file. So if I have queue: myQueue in the configuration file, the actual name of the queue in AWS must be myQueue.fifo.

This also led me to understand that the SQS queues should be of the FIFO type, instead of the Standard one. This is also not mentioned anywhere in the documentation.

Hi,

Can you add localization support for templating with Faker, define by an environment variable ?
I didn't see how to change it on the documentation of Mockintosh and I saw that Faker is init by default.

Here my two cents for this issue, I tried it locally and change the locale by defining the environment variable in my container.
Works well, give it a try :)

diff --git a/mockintosh/templating.py b/mockintosh/templating.py
index 22c2c2d..2aa6631 100644
--- a/mockintosh/templating.py
+++ b/mockintosh/templating.py
@@ -26,7 +26,7 @@ if not cov_no_import:
     import queue
 
 compiler = Compiler()
-faker = Faker()
+faker = Faker(environ.get('MOCKINTOSH_FAKER_LOCALE','en_US'))
 hbs_faker = HbsFaker()
 
 debug_mode = environ.get('MOCKINTOSH_DEBUG', False)

Thanks for this wonderful app 👍

I was testing out mockintosh for our enterprise - so, I downloaded the petstore swagger json example - Link here - and then tried to use the steps mockintosh provided to create automatic mocks.

I found that the created mocks does not provide complete functionality. When I did GET /pet/:petID, what I got was a dictionary of strings - wherein in reality, it was a dictionary of strings, arrays, and other dictionary.

Please fix the same. While I love the configuration driven (rather than UI) method followed by mockintosh, I am unable to recommend mockintosh because of this rather fundamental error. The question came that - this is too new a product, and if it cannot handle this, we do not know what all else might fail later.

Hi 👋

Is it possible to have Traffic Logs and Unhandled Requests logging enabled from mockintosh startup. I.e. to configure them via environment varibales?

In my setup Mockintosh is part of docker-compose config. What I need is to log all mock traffic inside docker right after calling docker-compose up.

(also submitted in stackoverflow if GH issues is not a place for usage questions. Slack group is apparently private)