Date   
Re: VPP api python scripts in packages for out-of-tree VPP plugins.

Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

It took some time, but https://gerrit.fd.io/r/20595

just got merged. Let me know if it does not work for you.

 

Vratko.

 

From: vpp-api-dev@... <vpp-api-dev@...> On Behalf Of Luca Muscariello via Lists.Fd.Io
Sent: Tuesday, 2019-July-09 15:56
To: vpp-api-dev@...; hicn-dev@...
Subject: [vpp-api-dev] VPP api python scripts in packages for out-of-tree VPP plugins.

 

Hi,

 

 

I have a use case to share:

 

A VPP plugin that is developed out of tree and exposes a binary API,

needs a few python scripts to generate language bindings for C, C++ etc.

 

vpp/plain/src/vpp-api/vapi/

vapi_c_gen.py

vapi_json_parser.py

vapi_cpp_gen.py

 

These scripts are not available in any deb/rpm archive currently.

 

As a countermeasure we have to retrieve the scripts from the web repo.

 

 

https://github.com/FDio/hicn/blob/master/hicn-plugin/CMakeLists.txt#L210-L212

 

 

It makes the cmake file very ugly.

For out-of-tree plugins this is very common as they can be built using vpp-dev/lib only.

 

Is there any work we can help with to get these scripts into a package, e.g. vpp-dev?

 

Thanks

Luca

 

 

flag day to enable API flag day process

Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

The main API flag document: [0].

 

I have most of implementation ready,

see [1] and comments there.

 

The missing parts is e-mail alerting

and postponing main vpp verify jobs

until API check passes (older stable/ VPP branches

are tricky to support in that way).

 

Please review, I would like to merge very soon.

 

Keeping CSIT list updated without automated

verify jobs takes some effort,

and it is pointless to add CRC checking only into CSIT

if VPP is not gated against API changes.

That is why both CSIT and ci-management parts

have to be merged together.

 

I will add the missing features

once the new verification is enabled.

 

Vratko.

 

[0] https://gerrit.fd.io/r/18356

[1] https://gerrit.fd.io/r/20708

Re: Question: getting the state of each nodes via Binary API

Hiroki Shirokura
 

Hi, 

I found my mistake.
I talked about node-state, but I really want to talk about process-state.
any-wait, done, time-wait aren't node-state, these are process-state. 

So, my question is "getting the state of each process via Binary API".
sorry. Do you know that such api was already exist too?

Best regard.
Hiroki Shirokura / slankdev

Question: getting the state of each nodes via Binary API

Hiroki Shirokura
 

Hi.
 
I'm trying to integrate vpp with another daemon using bin-api.
I want to get the state information of each node via VPP's bin-api.
In my understand, there is no api to get the state of each nodes (we can get only node-index by node-name), 
 
we can get them via VPP's cli such as following. 

DBGvpp# show runtime
Time 7.3, average vectors/node 0.00, last 128 main loops 0.00 per node 0.00
  vector rates in 0.0000e0, out 0.0000e0, drop 0.0000e0, punt 0.0000e0
             Name                 State         Calls          Vectors        Suspends         Clocks       Vectors/Call
api-rx-from-ring                any wait                 0               0               1          5.39e6            0.00
...(snip)...
send-rs-process                 any wait                 0               0               1          3.89e3            0.00
startup-config-process            done                   1               0               1          4.22e8            0.00
statseg-collector-process       time wait                0               0               1          1.89e6            0.00
unix-cli-stdin                   active                  0               0              39          9.06e7            0.00
...(snip)...

I need to get the STATE (such as any-wait, time-wait, .. or done) via bin-api. 
 
Do you know that such api was already exist?
If there is no such api, I'll send a patch.
 
Best regard.
Hiroki Shirokura / slankdev

Re: RFC: New model for stream messages

Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

+1 in general, but we should be careful
with respect to clients.

Perhaps choose a suffix different from _dump,
so users can infer the return value structure more easily?

Vratko.

-----Original Message-----
From: vpp-api-dev@... <vpp-api-dev@...> On Behalf Of Dave Wallace
Sent: Tuesday, 2019-June-18 16:12
To: vpp-api-dev@...
Cc: Dave Barach (dbarach) <@vppguy>
Subject: Re: [vpp-api-dev] RFC: New model for stream messages

+1

On 6/18/2019 3:02 AM, Ole Troan wrote:
Hi,

Dave made a suggestion to get rid of the control_ping wrapper hack that we use to detect last message in a stream for dump/details.

We have implemented something in:
https://gerrit.fd.io/r/#/c/20191/

Instead of having to wrap dump/detail calls in control ping, send
details messages in between a normal reply / request pair. As expressed in the below service statement.

Example:

service {
rpc map_domain_dump returns map_domain_dump_reply
stream map_domain_details;
};

autoreply define map_domain_dump
{
u32 client_index;
u32 context;
};

define map_domain_details
{
u32 context;
u32 domain_index;
vl_api_ip6_prefix_t ip6_prefix;
vl_api_ip4_prefix_t ip4_prefix;
vl_api_ip6_prefix_t ip6_src;
u8 ea_bits_len;
u8 psid_offset;
u8 psid_length;
u8 flags;
u16 mtu;
string tag[limit=64];
};


It's backwards compatible, so only those messages that have the service definition will use the new model.
But of course all language bindings will have to support this.

Views?

Best regards,
Ole

Re: VPP api python scripts in packages for out-of-tree VPP plugins.

Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

Adding vpp-dev, as I am not sure who reads vpp-api-dev.

 

Here [0] is my attempt at a simple solution.

 

Vratko.

 

[0] https://gerrit.fd.io/r/20595

 

From: vpp-api-dev@... <vpp-api-dev@...> On Behalf Of Luca Muscariello via Lists.Fd.Io
Sent: Tuesday, 2019-July-09 15:56
To: vpp-api-dev@...; hicn-dev@...
Subject: [vpp-api-dev] VPP api python scripts in packages for out-of-tree VPP plugins.

 

Hi,

 

 

I have a use case to share:

 

A VPP plugin that is developed out of tree and exposes a binary API,

needs a few python scripts to generate language bindings for C, C++ etc.

 

vpp/plain/src/vpp-api/vapi/

vapi_c_gen.py

vapi_json_parser.py

vapi_cpp_gen.py

 

These scripts are not available in any deb/rpm archive currently.

 

As a countermeasure we have to retrieve the scripts from the web repo.

 

 

https://github.com/FDio/hicn/blob/master/hicn-plugin/CMakeLists.txt#L210-L212

 

 

It makes the cmake file very ugly.

For out-of-tree plugins this is very common as they can be built using vpp-dev/lib only.

 

Is there any work we can help with to get these scripts into a package, e.g. vpp-dev?

 

Thanks

Luca

 

 

VPP api python scripts in packages for out-of-tree VPP plugins.

Luca Muscariello
 

Hi,

 

 

I have a use case to share:

 

A VPP plugin that is developed out of tree and exposes a binary API,

needs a few python scripts to generate language bindings for C, C++ etc.

 

vpp/plain/src/vpp-api/vapi/

vapi_c_gen.py

vapi_json_parser.py

vapi_cpp_gen.py

 

These scripts are not available in any deb/rpm archive currently.

 

As a countermeasure we have to retrieve the scripts from the web repo.

 

 

https://github.com/FDio/hicn/blob/master/hicn-plugin/CMakeLists.txt#L210-L212

 

 

It makes the cmake file very ugly.

For out-of-tree plugins this is very common as they can be built using vpp-dev/lib only.

 

Is there any work we can help with to get these scripts into a package, e.g. vpp-dev?

 

Thanks

Luca

 

 

Re: RFC: New model for stream messages

Dave Wallace
 

+1

On 6/18/2019 3:02 AM, Ole Troan wrote:
Hi,

Dave made a suggestion to get rid of the control_ping wrapper hack that we use to detect last message in a stream for dump/details.

We have implemented something in:
https://gerrit.fd.io/r/#/c/20191/

Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal
reply / request pair. As expressed in the below service statement.

Example:

service {
rpc map_domain_dump returns map_domain_dump_reply
stream map_domain_details;
};

autoreply define map_domain_dump
{
u32 client_index;
u32 context;
};

define map_domain_details
{
u32 context;
u32 domain_index;
vl_api_ip6_prefix_t ip6_prefix;
vl_api_ip4_prefix_t ip4_prefix;
vl_api_ip6_prefix_t ip6_src;
u8 ea_bits_len;
u8 psid_offset;
u8 psid_length;
u8 flags;
u16 mtu;
string tag[limit=64];
};


It's backwards compatible, so only those messages that have the service definition will use the new model.
But of course all language bindings will have to support this.

Views?

Best regards,
Ole

RFC: New model for stream messages

Ole Troan
 

Hi,

Dave made a suggestion to get rid of the control_ping wrapper hack that we use to detect last message in a stream for dump/details.

We have implemented something in:
https://gerrit.fd.io/r/#/c/20191/

Instead of having to wrap dump/detail calls in control ping, send details messages in between a normal
reply / request pair. As expressed in the below service statement.

Example:

service {
rpc map_domain_dump returns map_domain_dump_reply
stream map_domain_details;
};

autoreply define map_domain_dump
{
u32 client_index;
u32 context;
};

define map_domain_details
{
u32 context;
u32 domain_index;
vl_api_ip6_prefix_t ip6_prefix;
vl_api_ip4_prefix_t ip4_prefix;
vl_api_ip6_prefix_t ip6_src;
u8 ea_bits_len;
u8 psid_offset;
u8 psid_length;
u8 flags;
u16 mtu;
string tag[limit=64];
};


It's backwards compatible, so only those messages that have the service definition will use the new model.
But of course all language bindings will have to support this.

Views?

Best regards,
Ole

Re: Some Proposed MAP-E/T API Additions

Jon Loeliger
 

On Mon, Dec 10, 2018 at 2:07 PM Ole Troan <otroan@...> wrote:
Jon,

On 10 Dec 2018, at 20:01, Jon Loeliger <jdl@...> wrote:

I did code it up, but I didn’t think the extra complexity was. 
 
Ah!  I missed that conclusion!  OK.
 
Let’s just reuse the number spaces.
Create error numbers in the map component itself. 

Sorry to be so difficult, but do you mean:

    - MAP errors in a space -1, -2, -3...
    - As part of map.h or as part of map.api
    - Something else?

We have user-visible functions that translate the API error return
codes into their strings and print them out to the user.  If we have
re-used number spaces across all the plugins because they are
placed in the *.api files, we will not have good error messages.

Cheers 
Ole_._,_.__,_

Thanks,
jdl

Re: Some Proposed MAP-E/T API Additions

Ole Troan
 

Jon,

On 10 Dec 2018, at 20:01, Jon Loeliger <jdl@...> wrote:

On Mon, Dec 10, 2018 at 12:30 PM Ole Troan <otroan@...> wrote:
Btw Jon,

Any views on global error return vs per module or per message?
(I was suggesting not to bother adding to the global errno list...)

Ole,

We (you and I) had mentioned per-plugin error numbers quite a while ago.
However, I don't know if anything ever came from that; subsequently, I
don't know how to encode a per-plugin API error number.

I did code it up, but I didn’t think the extra complexity was. 


I wasn't sure what to do with them based on your review comment.  Sorry.
I can remove them and just return a generic error and have he CLI report
some generic statement ("Those parameters won't work") or something.

Let’s just reuse the number spaces.
Create error numbers in the map component itself. 

Cheers 
Ole


_,_

Links:

You receive all messages sent to this group.

View/Reply Online (#13) | Reply To Group | Reply To Sender | Mute This Topic | New Topic

Your Subscription | Contact Group Owner | Unsubscribe [otroan@...]

_._,_._,_

Re: Some Proposed MAP-E/T API Additions

Jon Loeliger
 

On Mon, Dec 10, 2018 at 12:30 PM Ole Troan <otroan@...> wrote:
Btw Jon,

Any views on global error return vs per module or per message?
(I was suggesting not to bother adding to the global errno list...)

Ole,

We (you and I) had mentioned per-plugin error numbers quite a while ago.
However, I don't know if anything ever came from that; subsequently, I
don't know how to encode a per-plugin API error number.

I wasn't sure what to do with them based on your review comment.  Sorry.
I can remove them and just return a generic error and have he CLI report
some generic statement ("Those parameters won't work") or something.

jdl

Re: Some Proposed MAP-E/T API Additions

Ole Troan
 

Btw Jon,

Any views on global error return vs per module or per message?
(I was suggesting not to bother adding to the global errno list...)

Cheers
Ole

On 10 Dec 2018, at 18:18, Ole Troan <otroan@...> wrote:

Hi Jon,

How important is it to you or the group that the CLI and API are consistent?
Should the CLI commands and the API handlers map onto the same underlying
and common implementation function?
The CLI is purely meant for development/debugging. Although it might not always be used that way.

My view on it, which might not reflect anyone else's, is that we should only have debug cli for commands where it does not make sense to have an API. For the rest the cli should be built on top of the API.


Ole,

Gotcha. So I will resubmit the patch with these changes:

- API calls will always set their underlying values.
(In the case of the previous trinary values, bools will be used.)
- The MAP CLI commands will be modified to require all the parameters needed
to make the underlying API calls. (For example, the sec-check and sec-check-frag
commands will be combined into one CLI command only.)

That sound right?
that sounds perfect!

Cheers,
Ole

Re: Some Proposed MAP-E/T API Additions

Ole Troan
 

Hi Jon,

How important is it to you or the group that the CLI and API are consistent?
Should the CLI commands and the API handlers map onto the same underlying
and common implementation function?
The CLI is purely meant for development/debugging. Although it might not always be used that way.

My view on it, which might not reflect anyone else's, is that we should only have debug cli for commands where it does not make sense to have an API. For the rest the cli should be built on top of the API.


Ole,

Gotcha. So I will resubmit the patch with these changes:

- API calls will always set their underlying values.
(In the case of the previous trinary values, bools will be used.)
- The MAP CLI commands will be modified to require all the parameters needed
to make the underlying API calls. (For example, the sec-check and sec-check-frag
commands will be combined into one CLI command only.)

That sound right?
that sounds perfect!

Cheers,
Ole

Re: Some Proposed MAP-E/T API Additions

Jon Loeliger
 

On Mon, Dec 10, 2018 at 9:33 AM Ole Troan <otroan@...> wrote:


> On 10 Dec 2018, at 16:05, Jon Loeliger <jdl@...> wrote:
>
> How important is it to you or the group that the CLI and API are consistent?
> Should the CLI commands and the API handlers map onto the same underlying
> and common implementation function?

The CLI is purely meant for development/debugging. Although it might not always be used that way.

My view on it, which might not reflect anyone else's, is that we should only have debug cli for commands where it does not make sense to have an API. For the rest the cli should be built on top of the API.


Ole,

Gotcha.  So I will resubmit the patch with these changes:

   - API calls will always set their underlying values.
      (In the case of the previous trinary values, bools will be used.)
   - The MAP CLI commands will be modified to require all the parameters needed
     to make the underlying API calls.   (For example, the sec-check and sec-check-frag
     commands will be combined into one CLI command only.)

That sound right?

Thanks,
jdl

Re: Some Proposed MAP-E/T API Additions

Ole Troan
 

On 10 Dec 2018, at 16:05, Jon Loeliger <jdl@...> wrote:

How important is it to you or the group that the CLI and API are consistent?
Should the CLI commands and the API handlers map onto the same underlying
and common implementation function?
The CLI is purely meant for development/debugging. Although it might not always be used that way.

My view on it, which might not reflect anyone else's, is that we should only have debug cli for commands where it does not make sense to have an API. For the rest the cli should be built on top of the API.

Cheers
Ole

Re: Some Proposed MAP-E/T API Additions

Jon Loeliger
 

On Mon, Dec 10, 2018 at 2:30 AM Ole Troan <otroan@...> wrote:
Dear Jon,

> My question is fundamentally, how do you want to do handle a binary value that
> may or may not need to be set?   So, for example, there is a boolean value, that
> may also need to be left "untouched/unchanged".   Do you want:

My thinking was to not allow for something being unchanged.

That is fine.  But the vppctl CLI will then have to change.  It currently is able to
set just one of those fields at a time.  We *can* allow the CLI to remain the same,
(ie, able to set just one field at a time), and have the API always set both fields,
but then there is a slight mis-alignment between the API and the CLI.  I was
trying to avoid that difference.

The dataplane is a slave to the control plane, it should do whatever the control plane tells it to do.

Of course.
 
And the control plane should know what values are already set and know the values it wants.

Isn’t this viable you think, especially now with the fine granularity you put in the API messages?

the options are:

1) Control plane knows all, and sets all values in API call

Which is what I think you just asked for.  But -- within each parameter grouping still.

2) Separate API call per knob

...and that seems like overkill here.

3) Trinary values

Which is what I implemented in the suggested patch.

4) Separate bools to indicate which knobs are “valid” (don’t like that one)

Agreed.  Seems clunky.
 
5) Defaults included in API

Not defaults, but rather current values.  Not sure how this would work...

Currently I’m trying to push for #1. But I might be proven that it is not possible in all cases...

It is possible.  But it either changes the current vppctl CLI commands, or it
has to allow for a difference in behavior between the API and the CLI.
(The API would alway set both, (eg, "sec-check-enable" and "sec-check-fragments"),
while the CLI would allow setting just one or the other at a time.)

How important is it to you or the group that the CLI and API are consistent?
Should the CLI commands and the API handlers map onto the same underlying
and common implementation function?

HTH,
jdl

Re: Some Proposed MAP-E/T API Additions

Ole Troan
 

Dear Jon,

My question is fundamentally, how do you want to do handle a binary value that
may or may not need to be set? So, for example, there is a boolean value, that
may also need to be left "untouched/unchanged". Do you want:
My thinking was to not allow for something being unchanged.
The dataplane is a slave to the control plane, it should do whatever the control plane tells it to do.
And the control plane should know what values are already set and know the values it wants.

Isn’t this viable you think, especially now with the fine granularity you put in the API messages?

the options are:

1) Control plane knows all, and sets all values in API call
2) Separate API call per knob
3) Trinary values
4) Separate bools to indicate which knobs are “valid” (don’t like that one)
5) Defaults included in API

Currently I’m trying to push for #1. But I might be proven that it is not possible in all cases...


- A boolean indicating that the associated boolean should be set?:

bool set_enable; /* t = Set the Enable/Disable value, f = ignore */
bool enable; /* If set_enable = t, set enable to this T/F value, else ignore */

- I had used a trinary value:

u8 enable; /* 0 = disable, 1 = enable, 0xFF = ignore */

- As I see it, you were not happy with the latter. Would the former be better?
Luckily my happiness isn’t a requirement. ;-)

Best regards,
Ole

Re: Some Proposed MAP-E/T API Additions

Jon Loeliger
 

On Sun, Dec 9, 2018 at 8:04 AM Ole Troan <otroan@...> wrote:
> I have posted a new patch that adds several new API calls
> that may be used to set MAP-E/MAP-T parameters.
>
> https://gerrit.fd.io/r/#/c/16397/
>
> If you are so inclined, review comments welcome!

Thanks a lot Jon. Added a few comments to the patch.

Awesome!  Thanks!

My question is fundamentally, how do you want to do handle a binary value that
may or may not need to be set?   So, for example, there is a boolean value, that
may also need to be left "untouched/unchanged".   Do you want:

    - A boolean indicating that the associated boolean should be set?:

        bool set_enable;       /* t = Set the Enable/Disable value, f = ignore */
        bool enable;             /* If set_enable = t, set enable to this T/F  value, else ignore */

    - I had used a trinary value:

      u8 enable;         /* 0 = disable, 1 = enable, 0xFF = ignore */

- As I see it, you were not happy with the latter.  Would the former be better?

Specifically, there are CLI commands where one of two boolean values are set,
but logically, the two boolean variables should be "grouped" into one API call.
Do you want one API call with a single boolean value per set-able value?

Or would you like 4 boolean values like this:

     bool set _enable;
     bool enable;
     bool set_other_bool;
     bool other_bool;

This applies, for example, to the security-check values:

/** \brief Set MAP security-check parameters
    @param client_index - opaque cookie to identify the sender
    @param context - sender context, to match reply w/ request
    @param enable - 1=enable security check on first inbound packet, ~0=untouched
    @param fragments - 1=enable check on (subsequent) fragments too, ~0=untouched
*/
autoreply define map_param_set_security_check
{
  u32 client_index;
  u32 context;
  u8 enable;
  u8 fragments;
};

Thanks,
jdl

Thanks,
jdl

Re: Some Proposed MAP-E/T API Additions

Ole Troan
 

I have posted a new patch that adds several new API calls
that may be used to set MAP-E/MAP-T parameters.

https://gerrit.fd.io/r/#/c/16397/

If you are so inclined, review comments welcome!
Thanks a lot Jon. Added a few comments to the patch.

Cheers,
Ole